WO2016015965A1 - Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle - Google Patents

Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle Download PDF

Info

Publication number
WO2016015965A1
WO2016015965A1 PCT/EP2015/065603 EP2015065603W WO2016015965A1 WO 2016015965 A1 WO2016015965 A1 WO 2016015965A1 EP 2015065603 W EP2015065603 W EP 2015065603W WO 2016015965 A1 WO2016015965 A1 WO 2016015965A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
motor vehicle
color image
value
snow
Prior art date
Application number
PCT/EP2015/065603
Other languages
French (fr)
Inventor
Patrick Eoghan Denny
Mark Patrick Griffin
Original Assignee
Connaught Electronics Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connaught Electronics Ltd. filed Critical Connaught Electronics Ltd.
Publication of WO2016015965A1 publication Critical patent/WO2016015965A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Definitions

  • the invention relates to a method for operating a camera system of a motor vehicle, in which a color image of an environmental region of the motor vehicle is provided by means of a camera of the camera system and pixels of the color image are selected for identifying snow in the color image, which have a brightness value, which is greater than a predetermined brightness limit value and/or which have a color value, which corresponds to a predetermined color value.
  • the invention relates to a driver assistance system for a motor vehicle as well as to a motor vehicle with a driver assistance system.
  • a method for identifying snow is disclosed.
  • the method is based on the principle of change detection, wherein it is examined if the value of a pixel in the first image has significantly changed compared to the value of the pixel in a second image captured later in time.
  • an object in the image such as for example snow in the form of falling snowflakes is assumed.
  • this object is solved by a method, by a driver assistance system as well as by a motor vehicle having the features according to the respective independent claims.
  • a method according to the invention for operating a camera system of a motor vehicle in which a color image of an environmental region of the motor vehicle is provided by means of a camera of the camera system and pixels of the color image are selected for identifying snow in the color image, which have a brightness value, which is greater than a predetermined brightness limit value, and/or which have a color value corresponding to a predetermined color value, according to the invention, it is provided that for at least one of the selected pixels, a weighting function is determined for identifying snow, wherein the weighting function is determined depending on a position of the at least one selected pixel in the color image.
  • the method according to the invention it thus becomes possible to make a particularly precise statement based on the position of the selected pixel in the color image whether or not the selected pixel is a pixel of the snow.
  • the selected pixels are therefore pixels, which can represent snow based on their brightness value and/or their color value.
  • the position in the color image is therefore taken into account because with known orientation of the camera, the probability for the snow is different in the different positions in the color image.
  • the probability can be empirically determined.
  • the weighting function can be determined. Based on the weighting function, thus, a statement can be made on how probable or with which reliability the selected pixel in the respective position describes snow in the environmental region.
  • the weighting function is determined depending on whether the position of the at least one selected pixel is disposed in a predetermined area of the color image.
  • This has the advantage that selected pixels in the predetermined area of the color image can for example be excluded from the further procedure or cannot be taken into account.
  • an area presents parts of the motor vehicle, for example a bumper, in an image.
  • this can occur with cameras with a fish-eye lens and/or the rearview cameras.
  • the predetermined area describes an area of the color image different from the motor vehicle.
  • snow lying on a bumper or another part of the motor vehicle is not taken into account.
  • the weighting function is determined depending on a number of further selected pixels having a predetermined distance to the at least one pixel.
  • a cluster of the selected pixels in an area with a predetermined size can be included in the weighting function.
  • clusters of the selected pixels can be taken into account in the weighting function.
  • This has the advantage that a stand-alone selected pixel can be assessed with a lower weight than a selected pixel having other selected pixels in the predetermined distance.
  • this can be advantageous because the snow usually does not appear punctually, but two-dimensionally in the environmental region and thus in the color image.
  • the weighting function is additionally determined depending on at least one environmental parameter provided by a motor vehicle side acquisition device and describing the environmental region of the motor vehicle. It is advantageous that additional information about the environmental region can be provided with the motor vehicle side acquisition device. Thus, the identification of snow can be particularly precisely and reliably effected thereby.
  • the acquisition device can include diverse sensors and/or information receivers. The information or the environmental parameters can then be included in the weighting function and support a statement whether or not the selected pixel is snow.
  • a further advantage of the use of environmental parameters is in that the information fusion occurring with the environmental parameters and the color image is able to increase the reliability of the identification of snow.
  • a current temperature in the environmental region of the motor vehicle is provided as the at least one environmental parameter.
  • the current temperature can for example be provided by means of a temperature sensor of the motor vehicle.
  • the consideration of the current temperature is advantageous because thereby a further environmental parameter can be included in the weighting function.
  • the infrared signal can be provided by means of an infrared sensor of the motor vehicle.
  • the infrared signal is determined in a spectral range between 780 nanometers and 1 millimeter of wavelength. Based on the infrared signal, a temperature of an object in the environmental region can be inferred.
  • the reflectivity of snow in the range of near infrared thus in the range of 0.78 micrometers to 3 micrometers of wavelength, can greatly decrease due to the grain size of snow compared to the range of the visible light.
  • weather information describing the environmental region which is received by means of the acquisition device from an external transmitting device, is provided as the at least one environmental parameter.
  • the weather information can for example be a current weather forecast, which is provided by an external transmitting device, for example a radio station and/or a source from the Internet.
  • this information from the Internet can for example be obtained very comfortably by means of an RSS feed.
  • the weather information can be obtained matched to the current whereabouts of the motor vehicle.
  • this can for example be achieved with a GNSS or a global navigation satellite system, for example GPS and/or GLONASS.
  • the GNSS system thus, the current whereabouts of the motor vehicle can be determined and the weather information accordingly adapted to the environmental region can be retrieved or obtained.
  • the weather information can for example include if snow is present and/or if snowfall is announced and/or if snow has fallen in the past days.
  • an approximated value for the temperature of the environmental region can also be taken into account based on the weather information.
  • an altitude of the motor vehicle with respect to a certain altitude reference point is provided as the at least one environmental parameter.
  • the altitude of the motor vehicle can for example be determined by means of a GNSS receiver and/or a barometer.
  • the altitude reference point can for example be the Amsterdam Ordnance Datum or the reference ellipsoid WGS 84 in the case of a GPS measurement. Due to the altitude, the probability of snow increasing with increasing altitude can be included in the weighting function.
  • the at least one environmental parameter is transmitted by a traction control system of the motor vehicle to the acquisition device.
  • the traction control system is also referred to as traction control and ensures that the wheels do not spin upon acceleration of the motor vehicle.
  • the traction control system is to prevent one or more wheels from spinning and the motor vehicle from laterally braking away with poor roadway conditions such as for example ice and/or snow on the roadway.
  • information of the automatic steering intervention of for example a steering tilt control system of the motor vehicle can also be used, which is activated if the traction control system detects ice or snow.
  • active control of the front wheel steering angle is used to maintain the stability of the motor vehicle.
  • the information from the traction control system can for example be tapped from a CAN bus of the motor vehicle.
  • an adaptation of camera parameters of the camera is performed upon identification of snow in the color image.
  • presetting of camera parameters can be started for example by means of a control unit of the motor vehicle.
  • the camera parameters can be accordingly adapted to the situation.
  • better image quality of the color image can be achieved.
  • the color image can thus be captured with a particularly high quality by the camera, because the camera parameters can be adapted to the special situation existing due to snow.
  • the special situation due to snow is for example a high reflection of light, in particular sunlight, which results in the danger of over-exposure of the color image being able to exist.
  • a white balance value of the camera and/or a light sensitivity value of the camera and/or a gamma value of the camera and/or a contrast value of the camera and/or a dynamic range control unit of the camera are adapted as the camera parameters.
  • the light sensitivity value can for example be a sensitivity value of the sensor of the camera, which is usually indicated in ISO. However, the light incidence on the sensor can also be reduced for example by means of an aperture of the camera.
  • the gamma value stands for a compensation factor of an imaging system for the perception of the human eye.
  • gamma correction is employed, which transforms a correction function for converting a physically proportional, that is linearly increasing, quantity into a quantity non-linearly increasing according to the human perception.
  • the dynamic range control unit controls the contrast of the color image among other things.
  • a high contrast image or an image with a high dynamic range is a digital image reproducing great brightness differences in much detail.
  • a function of the camera in particular automatic calibration, is deactivated upon identification of snow in the color image.
  • the automatic calibration of the camera can also belong to this.
  • the identification of snow is performed exclusively if the ambient temperature falls below a predetermined temperature limit value.
  • the method for identifying snow can be deactivated until it is fallen below the predetermined temperature limit value.
  • highly improbable ambient temperatures for example ambient temperatures above 5 C or above 10 °C or in particular above 15 ⁇ , can be suppressed or avoide d.
  • resources of the motor vehicle can be saved by the temporary deactivation of the method for identifying snow depending on the ambient temperature.
  • a motor vehicle according to the invention in particular a passenger car, includes a driver assistance system according to the invention.
  • Fig. 1 in schematic plan view an embodiment of a motor vehicle according to the invention with a driver assistance system including a camera system;
  • Fig. 2 a color image of an environmental region of the motor vehicle, which is captured by means of a camera of the camera system.
  • Fig. 1 a plan view of a motor vehicle 1 with a driver assistance system 2 according to an embodiment of the invention is schematically illustrated.
  • the driver assistance system 2 includes a camera system 3 and an acquisition device 4.
  • the camera system 3 further includes a camera 5 and an evaluation unit 6.
  • the camera 5 can be a CMOS camera or else a CCD camera or any image capturing device, by which color images of an environmental region 7 of the motor vehicle 1 can be captured.
  • the camera 5 is disposed in a region behind a windshield 8 of the motor vehicle 1 and oriented to the front in direction of travel of the motor vehicle 1 .
  • the arrangement of the camera 5 on the motor vehicle 1 is variously possible, thus, the camera 5 can for example also be disposed on a front or a rear or laterally on the motor vehicle 1 .
  • several of the cameras 5 are also preferably provided, which capture the environmental region 7.
  • an acquisition device 4 is schematically shown.
  • the acquisition device 4 can also be arbitrarily disposed on the motor vehicle 1 .
  • the acquisition device 4 can include a temperature sensor, by which a current temperature in the environmental region 7 of the motor vehicle 1 can be acquired.
  • the acquisition device 4 can for example include an infrared sensor, by which an infrared image of the environmental region 7 can be recorded.
  • the acquisition device 4 can have a GNSS receiver, thus a receiver for the signals of a global navigation satellite system. Based on the GNSS receiver, the current position of the motor vehicle 1 and/or the altitude, on which the motor vehicle 1 is currently located, can be acquired. Furthermore, the acquisition device 4 can have an Internet receiver and/or a radio receiver and/or a television channel receiver. In this manner, weather information for the environmental region 7 can be received. In addition, the acquisition device 4 can be adapted to receive data from a traction control system of the motor vehicle 1 .
  • the flow of the method for identifying snow is as follows according to an embodiment of the invention.
  • a color image 9 of the environmental region 7 is provided.
  • pixels 10 are selected, which have a brightness value, which is greater than a predetermined brightness limit value.
  • the color image is for example converted into a YUV color space, where the Y channel of the YUV color space describes the brightness of the respective pixel 10 of the color image 9.
  • the pixel 10 is selected if it has a color value corresponding to a predetermined color value. This can be mathematically represented as follows:
  • a pixel 1 1 of the color image 9 is a selected pixel 10 if the value for the Y channel is less than or equal to a predetermined brightness limit value and the respective values for an R channel, a G channel and a B channel substantially correspond to each other.
  • the R channel, the G channel and the B channel are respective channels of the color image 9 in an RGB color space.
  • the at least one selected pixel 10 is subsequently examined with a weighting function W s for identifying snow 12 in the environmental region 7.
  • the weighting function W s can for example be represented as follows:
  • W s A * T V + B * P r + C * IR V + D * Z V + E * TS V , wherein T v is a current temperature in the environmental region 7, P r is a position of the at least one selected pixel 10, IR V is an intensity of an infrared signal, which describes the environmental region 7, Z v is an altitude of the motor vehicle 1 with respect to a certain altitude reference point and TS V is a parameter of a traction control system of the motor vehicle 1 or of a steering tilt control system.
  • A is the coefficient of the current temperature T v
  • B is the coefficient of the position P r
  • C is the coefficient of the intensity of the infrared signal IR V
  • D is the coefficient of the altitude Z v
  • E is the coefficient of the parameter of a traction control system TS V .
  • the environmental parameters T v , IR V , Z v , TS V can be weighted corresponding to their information.
  • the value of the coefficients or the weighting can for example be empirically determined.
  • a meaningful amount of training data can be used for selecting a value of the respective coefficient A, B, C, D, E.
  • snow can be determined in the environmental region 7.
  • the snow 12 could be found or determined if the weighting function W s is above a predetermined limit value.
  • camera parameters of the camera can be adapted to the environmental region 7 with the snow 1 1 by means of a control unit of the motor vehicle 1 .
  • a white balance value and/or a light sensitivity value and/or a gamma value and/or a contrast value and/or a dynamic range control unit of the camera can be adapted as the camera parameters.
  • the white balance the camera 5 is adjusted to a respective light situation.
  • the light sensitivity value of the camera 5 can for example be an ISO value, which gives information on how sensitive a sensor, for example a CCD array, of the camera 5 is with respect to light, thus the incidence of photons.
  • the camera 5 has a too high light sensitivity value, there is the risk of over-exposure, in particular if the snow 12 more intensely reflects light.
  • the optimum adaptation of the light sensitivity value is reasonable because in case of a too high light sensitivity or a too high light sensitivity value, the noise in the color image can be increased or amplified.
  • the gamma value can contribute to convert or transform a linear brightness progression into a non-linear brightness progression, which is more comfortable to the human eye.
  • the contrast value can ensure that a range of values of the color image 9 is optimally utilized.
  • the dynamic range control unit is provided for capturing the color image 9 as high in quality as possible with all of the details of the color image 9.
  • an automatic calibration of the camera 5 is deactivated if the snow 12 is identified. Additionally or alternatively, it is provided that the identification of the snow 12 is performed exclusively if the ambient temperature T v falls below a predetermined temperature limit value.
  • the identification of the snow 12 is performed exclusively if the ambient temperature T v falls below a predetermined temperature limit value.
  • Fig. 2 shows the color image 9 with the environmental region 7, in which the snow 12 is located centrally in the color image 9.
  • the color image 9 is divided in the pixels 1 1 .
  • the pixels 10 selected from the pixels, which have the brightness value, which is greater than the predetermined brightness limit value, and have the color value corresponding to the predetermined color value, are indicated in the position in the color image 9, where the snow 12 is located.

Abstract

The invention relates to a method for operating a camera system (3) of a motor vehicle (1), in which a color image (9) of an environmental region (7) of the motor vehicle (1) is provided by means of a camera (5) of the camera system (3) and pixels (11) of the color image (9) are selected for identifying snow (12) in the color image (9), which have a brightness value (Y), which is greater than a predetermined brightness limit value, and/or which have a color value (R, B, G) corresponding to a predetermined color value, wherein for identifying snow, (12) for at least one of the selected pixels (10), a weighting function (Ws) is determined, wherein the weighting function (Ws) is determined depending on a position (Pr) of the at least one selected pixel (10) in the color image (9).

Description

Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle
The invention relates to a method for operating a camera system of a motor vehicle, in which a color image of an environmental region of the motor vehicle is provided by means of a camera of the camera system and pixels of the color image are selected for identifying snow in the color image, which have a brightness value, which is greater than a predetermined brightness limit value and/or which have a color value, which corresponds to a predetermined color value. In addition, the invention relates to a driver assistance system for a motor vehicle as well as to a motor vehicle with a driver assistance system.
Methods for operating a camera system of a motor vehicle, in which snow is identified in an image, are known from the prior art. Thus, in US 8,285,059 B2, a method is described, which uses a brightness value of a YUV color space in the image to identify snow. In addition, it is assumed that the maximums of a respective histogram of an R channel, a G channel and a B channel of an RGB color space are close to each other in order to identify snow. In addition, GPS information is used, which is to allow statement on how probable the location of capture of the image is for snow.
In US 8,582,809 B2 too, a method for identifying snow is disclosed. The method is based on the principle of change detection, wherein it is examined if the value of a pixel in the first image has significantly changed compared to the value of the pixel in a second image captured later in time. Upon change of the pixel, an object in the image such as for example snow in the form of falling snowflakes is assumed.
The mentioned methods are disadvantageous in that these methods operate almost exclusively with techniques of image processing and the reliability thereof is therefore subject to restrictions.
It is the object of the invention to provide a method, a driver assistance system as well as a motor vehicle, by which snow in an environmental region of the motor vehicle can be more reliably identified.
According to the invention, this object is solved by a method, by a driver assistance system as well as by a motor vehicle having the features according to the respective independent claims. In a method according to the invention for operating a camera system of a motor vehicle, in which a color image of an environmental region of the motor vehicle is provided by means of a camera of the camera system and pixels of the color image are selected for identifying snow in the color image, which have a brightness value, which is greater than a predetermined brightness limit value, and/or which have a color value corresponding to a predetermined color value, according to the invention, it is provided that for at least one of the selected pixels, a weighting function is determined for identifying snow, wherein the weighting function is determined depending on a position of the at least one selected pixel in the color image.
By the method according to the invention, it thus becomes possible to make a particularly precise statement based on the position of the selected pixel in the color image whether or not the selected pixel is a pixel of the snow. The selected pixels are therefore pixels, which can represent snow based on their brightness value and/or their color value. The position in the color image is therefore taken into account because with known orientation of the camera, the probability for the snow is different in the different positions in the color image. For example, the probability can be empirically determined. Depending on the probability, the weighting function can be determined. Based on the weighting function, thus, a statement can be made on how probable or with which reliability the selected pixel in the respective position describes snow in the environmental region.
Furthermore, it is provided that the weighting function is determined depending on whether the position of the at least one selected pixel is disposed in a predetermined area of the color image. This has the advantage that selected pixels in the predetermined area of the color image can for example be excluded from the further procedure or cannot be taken into account. Thus, it can for example be the case that an area presents parts of the motor vehicle, for example a bumper, in an image. For example, this can occur with cameras with a fish-eye lens and/or the rearview cameras. Thus, it can be provided that the predetermined area describes an area of the color image different from the motor vehicle. Thus, it is possible that for example snow lying on a bumper or another part of the motor vehicle, is not taken into account.
Preferably, the weighting function is determined depending on a number of further selected pixels having a predetermined distance to the at least one pixel. This means that a cluster of the selected pixels in an area with a predetermined size can be included in the weighting function. Thus, clusters of the selected pixels can be taken into account in the weighting function. This has the advantage that a stand-alone selected pixel can be assessed with a lower weight than a selected pixel having other selected pixels in the predetermined distance. Furthermore, this can be advantageous because the snow usually does not appear punctually, but two-dimensionally in the environmental region and thus in the color image.
In particular, it is provided that the weighting function is additionally determined depending on at least one environmental parameter provided by a motor vehicle side acquisition device and describing the environmental region of the motor vehicle. It is advantageous that additional information about the environmental region can be provided with the motor vehicle side acquisition device. Thus, the identification of snow can be particularly precisely and reliably effected thereby. The acquisition device can include diverse sensors and/or information receivers. The information or the environmental parameters can then be included in the weighting function and support a statement whether or not the selected pixel is snow. A further advantage of the use of environmental parameters is in that the information fusion occurring with the environmental parameters and the color image is able to increase the reliability of the identification of snow.
Preferably, it is provided that a current temperature in the environmental region of the motor vehicle is provided as the at least one environmental parameter. The current temperature can for example be provided by means of a temperature sensor of the motor vehicle. The consideration of the current temperature is advantageous because thereby a further environmental parameter can be included in the weighting function. Usually, the probability of snow considerably decreases with increasing temperature above 0 Ό due to physical laws.
Furthermore, it is provided that an intensity of an infrared signal describing the
environmental region is provided as the at least one environmental parameter. For example, the infrared signal can be provided by means of an infrared sensor of the motor vehicle. Usually, the infrared signal is determined in a spectral range between 780 nanometers and 1 millimeter of wavelength. Based on the infrared signal, a temperature of an object in the environmental region can be inferred. However, the reflectivity of snow in the range of near infrared, thus in the range of 0.78 micrometers to 3 micrometers of wavelength, can greatly decrease due to the grain size of snow compared to the range of the visible light. Furthermore, it is provided that weather information describing the environmental region, which is received by means of the acquisition device from an external transmitting device, is provided as the at least one environmental parameter. Thus, the weather information can for example be a current weather forecast, which is provided by an external transmitting device, for example a radio station and/or a source from the Internet. Thus, this information from the Internet can for example be obtained very comfortably by means of an RSS feed. Thus, the weather information can be obtained matched to the current whereabouts of the motor vehicle. Thus, this can for example be achieved with a GNSS or a global navigation satellite system, for example GPS and/or GLONASS. By the GNSS system, thus, the current whereabouts of the motor vehicle can be determined and the weather information accordingly adapted to the environmental region can be retrieved or obtained. Thus, the weather information can for example include if snow is present and/or if snowfall is announced and/or if snow has fallen in the past days. Furthermore, an approximated value for the temperature of the environmental region can also be taken into account based on the weather information.
Preferably, it is provided that an altitude of the motor vehicle with respect to a certain altitude reference point is provided as the at least one environmental parameter. The altitude of the motor vehicle can for example be determined by means of a GNSS receiver and/or a barometer. The altitude reference point can for example be the Amsterdam Ordnance Datum or the reference ellipsoid WGS 84 in the case of a GPS measurement. Due to the altitude, the probability of snow increasing with increasing altitude can be included in the weighting function.
Furthermore, it is provided that the at least one environmental parameter is transmitted by a traction control system of the motor vehicle to the acquisition device. The traction control system is also referred to as traction control and ensures that the wheels do not spin upon acceleration of the motor vehicle. The traction control system is to prevent one or more wheels from spinning and the motor vehicle from laterally braking away with poor roadway conditions such as for example ice and/or snow on the roadway. For example, information of the automatic steering intervention of for example a steering tilt control system of the motor vehicle can also be used, which is activated if the traction control system detects ice or snow. In the steering tilt control system, active control of the front wheel steering angle is used to maintain the stability of the motor vehicle. The information from the traction control system can for example be tapped from a CAN bus of the motor vehicle. In particular, it is provided that upon identification of snow in the color image, an adaptation of camera parameters of the camera is performed. Thus, presetting of camera parameters can be started for example by means of a control unit of the motor vehicle. This means that if snow has been identified, the camera parameters can be accordingly adapted to the situation. Advantageously, thereby, better image quality of the color image can be achieved. The color image can thus be captured with a particularly high quality by the camera, because the camera parameters can be adapted to the special situation existing due to snow. The special situation due to snow is for example a high reflection of light, in particular sunlight, which results in the danger of over-exposure of the color image being able to exist.
It is provided that a white balance value of the camera and/or a light sensitivity value of the camera and/or a gamma value of the camera and/or a contrast value of the camera and/or a dynamic range control unit of the camera are adapted as the camera parameters. The light sensitivity value can for example be a sensitivity value of the sensor of the camera, which is usually indicated in ISO. However, the light incidence on the sensor can also be reduced for example by means of an aperture of the camera. The gamma value stands for a compensation factor of an imaging system for the perception of the human eye. Herein, gamma correction is employed, which transforms a correction function for converting a physically proportional, that is linearly increasing, quantity into a quantity non-linearly increasing according to the human perception. Furthermore, the dynamic range control unit controls the contrast of the color image among other things. Thus, a high contrast image or an image with a high dynamic range is a digital image reproducing great brightness differences in much detail. Thus, it is advantageous to again obtain a quality of the color image with the snow in the environmental region as high as possible.
Furthermore, it is provided that a function of the camera, in particular automatic calibration, is deactivated upon identification of snow in the color image. Thus, there are certain functions of the camera, which cannot be correctly executed due to the presence of snow. The automatic calibration of the camera can also belong to this. Thus, it is advantageous if for example the automatic calibration is deactivated or not used during the identification or the presence of snow, in order to prevent false automatic calibration.
In a further embodiment, it is provided that the identification of snow is performed exclusively if the ambient temperature falls below a predetermined temperature limit value. This means that the method for identifying snow can be deactivated until it is fallen below the predetermined temperature limit value. This is advantageous because thereby a possible false detection or false identification of snow with highly improbable ambient temperatures, for example ambient temperatures above 5 C or above 10 °C or in particular above 15 Ό, can be suppressed or avoide d. Furthermore, it is advantageous that resources of the motor vehicle can be saved by the temporary deactivation of the method for identifying snow depending on the ambient temperature.
A driver assistance system according to the invention for a motor vehicle includes at least one camera of a camera system and an evaluation unit of the camera system, which is adapted to perform a method according to the invention.
A motor vehicle according to the invention, in particular a passenger car, includes a driver assistance system according to the invention.
The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the driver assistance system according to the invention as well as to the motor vehicle according to the invention.
Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or alone, without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations.
Below, embodiments of the invention are explained in more detail based on schematic drawings. There show:
Fig. 1 in schematic plan view an embodiment of a motor vehicle according to the invention with a driver assistance system including a camera system; and
Fig. 2 a color image of an environmental region of the motor vehicle, which is captured by means of a camera of the camera system. In Fig. 1 , a plan view of a motor vehicle 1 with a driver assistance system 2 according to an embodiment of the invention is schematically illustrated. In the embodiment, the driver assistance system 2 includes a camera system 3 and an acquisition device 4. The camera system 3 further includes a camera 5 and an evaluation unit 6.
The camera 5 can be a CMOS camera or else a CCD camera or any image capturing device, by which color images of an environmental region 7 of the motor vehicle 1 can be captured.
In the embodiment according to Fig. 1 , the camera 5 is disposed in a region behind a windshield 8 of the motor vehicle 1 and oriented to the front in direction of travel of the motor vehicle 1 . However, the arrangement of the camera 5 on the motor vehicle 1 is variously possible, thus, the camera 5 can for example also be disposed on a front or a rear or laterally on the motor vehicle 1 . Furthermore, several of the cameras 5 are also preferably provided, which capture the environmental region 7.
In the present embodiment, an acquisition device 4 is schematically shown. The acquisition device 4 can also be arbitrarily disposed on the motor vehicle 1 . The acquisition device 4 can include a temperature sensor, by which a current temperature in the environmental region 7 of the motor vehicle 1 can be acquired. Additionally, the acquisition device 4 can for example include an infrared sensor, by which an infrared image of the environmental region 7 can be recorded.
Moreover, the acquisition device 4 can have a GNSS receiver, thus a receiver for the signals of a global navigation satellite system. Based on the GNSS receiver, the current position of the motor vehicle 1 and/or the altitude, on which the motor vehicle 1 is currently located, can be acquired. Furthermore, the acquisition device 4 can have an Internet receiver and/or a radio receiver and/or a television channel receiver. In this manner, weather information for the environmental region 7 can be received. In addition, the acquisition device 4 can be adapted to receive data from a traction control system of the motor vehicle 1 .
The flow of the method for identifying snow is as follows according to an embodiment of the invention. By means of the camera 5, a color image 9 of the environmental region 7 is provided. In the color image 9, pixels 10 are selected, which have a brightness value, which is greater than a predetermined brightness limit value. To this, the color image is for example converted into a YUV color space, where the Y channel of the YUV color space describes the brightness of the respective pixel 10 of the color image 9. Furthermore, the pixel 10 is selected if it has a color value corresponding to a predetermined color value. This can be mathematically represented as follows:
A pixel 1 1 of the color image 9 is a selected pixel 10 if the value for the Y channel is less than or equal to a predetermined brightness limit value and the respective values for an R channel, a G channel and a B channel substantially correspond to each other. Therein, the R channel, the G channel and the B channel are respective channels of the color image 9 in an RGB color space.
The at least one selected pixel 10 is subsequently examined with a weighting function Ws for identifying snow 12 in the environmental region 7. The weighting function Ws can for example be represented as follows:
Ws = A*TV + B*Pr + C*IRV + D*ZV + E*TSV, wherein Tv is a current temperature in the environmental region 7, Pr is a position of the at least one selected pixel 10, IRV is an intensity of an infrared signal, which describes the environmental region 7, Zv is an altitude of the motor vehicle 1 with respect to a certain altitude reference point and TSV is a parameter of a traction control system of the motor vehicle 1 or of a steering tilt control system. A is the coefficient of the current temperature Tv, B is the coefficient of the position Pr, C is the coefficient of the intensity of the infrared signal IRV, D is the coefficient of the altitude Zv and E is the coefficient of the parameter of a traction control system TSV.
With the coefficients A to E, thus, the environmental parameters Tv, IRV, Zv, TSV can be weighted corresponding to their information. The value of the coefficients or the weighting can for example be empirically determined. Thus, for example, a meaningful amount of training data can be used for selecting a value of the respective coefficient A, B, C, D, E.
Thus, based on the weighting function Ws, snow can be determined in the environmental region 7. Thus, it can be assumed that the snow 12 could be found or determined if the weighting function Ws is above a predetermined limit value.
If at least one of the environmental parameters Tv, IRV, Zv, TSV cannot be provided, thus, this value can for example be included in the weighting function Ws with zero. Depending on the result whether or not the snow 12 has been determined, camera parameters of the camera can be adapted to the environmental region 7 with the snow 1 1 by means of a control unit of the motor vehicle 1 . For example, a white balance value and/or a light sensitivity value and/or a gamma value and/or a contrast value and/or a dynamic range control unit of the camera can be adapted as the camera parameters. By the white balance, the camera 5 is adjusted to a respective light situation. The light sensitivity value of the camera 5 can for example be an ISO value, which gives information on how sensitive a sensor, for example a CCD array, of the camera 5 is with respect to light, thus the incidence of photons.
If the camera 5 has a too high light sensitivity value, there is the risk of over-exposure, in particular if the snow 12 more intensely reflects light. The optimum adaptation of the light sensitivity value is reasonable because in case of a too high light sensitivity or a too high light sensitivity value, the noise in the color image can be increased or amplified. The gamma value can contribute to convert or transform a linear brightness progression into a non-linear brightness progression, which is more comfortable to the human eye. For example, the contrast value can ensure that a range of values of the color image 9 is optimally utilized. The dynamic range control unit is provided for capturing the color image 9 as high in quality as possible with all of the details of the color image 9.
Furthermore, it can be provided that an automatic calibration of the camera 5 is deactivated if the snow 12 is identified. Additionally or alternatively, it is provided that the identification of the snow 12 is performed exclusively if the ambient temperature Tv falls below a predetermined temperature limit value. Hereby, it can be achieved that for example resources of the motor vehicle 1 , for example energy and/or computing capacity, are saved.
Fig. 2 shows the color image 9 with the environmental region 7, in which the snow 12 is located centrally in the color image 9. The color image 9 is divided in the pixels 1 1 . The pixels 10 selected from the pixels, which have the brightness value, which is greater than the predetermined brightness limit value, and have the color value corresponding to the predetermined color value, are indicated in the position in the color image 9, where the snow 12 is located.

Claims

Claims
1 . Method for operating a camera system (3) of a motor vehicle (1 ), in which a color image (9) of an environmental region (7) of the motor vehicle (1 ) is provided by means of a camera (5) of the camera system (3) and pixels (1 1 ) of the color image (9) are selected for identifying snow (12) in the color image (9), which have a brightness value (Y), which is greater than a predetermined brightness limit value, and/or which have a color value (R, G, B) corresponding to a predetermined color value,
characterized in that
for identifying snow (12), for at least one of the selected pixels (10), a weighting function (Ws) is determined, wherein the weighting function (Ws) is determined depending on a position (Pr) of the at least one selected pixel (10) in the color image (9).
2. Method according to claim 1 ,
characterized in that
the weighting function (Ws) is determined depending on whether the position (Pr) of the at least one selected pixel (10) is disposed in a predetermined area of the color image (9).
3. Method according to claim 1 or 2,
characterized in that
the weighting function (Ws) is determined depending on a number of further selected pixels (10) having a predetermined distance to the at least one pixel (10).
4. Method according to any one of the preceding claims,
characterized in that
the weighting function (Ws) is additionally determined depending on at least one environmental parameter (Tv, IRV, Zv, TSV) provided by a motor vehicle side acquisition device (4) and describing the environmental region (7) of the motor vehicle (1 ).
5. Method according to claim 4,
characterized in that
a current temperature (Tv) in the environmental region (7) of the motor vehicle (1 ) is provided as the at least one environmental parameter (Tv, IRV, Zv, TSV).
6. Method according to claim 4 or 5,
characterized in that
an intensity of an infrared signal (IRV) describing the environmental region (7) is provided as the at least one environmental parameter (Tv, IRV, Zv, TSV).
7. Method according to any one of claims 4 to 6,
characterized in that
weather information describing the environmental region (7), which is received by means of the acquisition device (4) from an external transmitting device, is provided as the at least one environmental parameter (Tv, IRV, Zv, TSV).
8. Method according to any one of claims 4 to 7,
characterized in that
an altitude of the motor vehicle (1 ) with respect to a certain altitude reference point is provided as the at least one environmental parameter (Tv, IRV, Zv, TSV).
9. Method according to any one of claims 4 to 8,
characterized in that
the at least one environmental parameter (Tv, IRV, Zv, TSV) is transmitted by a traction control system of the motor vehicle (1 ) to the acquisition device (4).
10. Method according to any one of the preceding claims,
characterized in that
upon identifying snow (12) in the color image (9), adaption of camera parameters of the camera (5) is performed.
1 1 . Method according to claim 10,
characterized in that
a white balance value of the camera (5) and/or a light sensitivity value of the camera (5) and/or a gamma value of the camera (5) and/or a contrast value of the camera (5) and/or a dynamic range control unit of the camera (5) are adapted as the camera parameters.
12. Method according to any one of the preceding claims,
characterized in that
upon identification of snow (12) in the color image (9), a function of the camera (5), in particular automatic calibration, is deactivated.
13. Method according to any one of the preceding claims,
characterized in that
the identification of snow (12) is performed exclusively if the ambient temperature (Tv) falls below a predetermined temperature limit value.
14. Driver assistance system (2) for a motor vehicle (1 ) including a camera (5) of a camera system (3) and an evaluation unit (6) of the camera system (3), which is adapted to perform a method according to the preceding claims.
15. Motor vehicle (1 ) including a driver assistance system (2) according to claim 14.
PCT/EP2015/065603 2014-07-29 2015-07-08 Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle WO2016015965A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102014110663.6 2014-07-29
DE102014110663.6A DE102014110663A1 (en) 2014-07-29 2014-07-29 Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle

Publications (1)

Publication Number Publication Date
WO2016015965A1 true WO2016015965A1 (en) 2016-02-04

Family

ID=53719754

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/065603 WO2016015965A1 (en) 2014-07-29 2015-07-08 Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle

Country Status (2)

Country Link
DE (1) DE102014110663A1 (en)
WO (1) WO2016015965A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102017208994A1 (en) 2017-05-29 2018-11-29 Audi Ag Method for determining result image data

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009027089A2 (en) * 2007-08-30 2009-03-05 Valeo Schalter Und Sensoren Gmbh Method and system for weather condition detection with image-based road characterization
US20100121577A1 (en) * 2008-04-24 2010-05-13 Gm Global Technology Operations, Inc. Three-dimensional lidar-based clear path detection
US20100296700A1 (en) * 2007-10-31 2010-11-25 Adc Automotive Distance Control Systems Gmbh Method and device for detecting the course of a traffic lane
US8285059B2 (en) 2008-05-20 2012-10-09 Xerox Corporation Method for automatic enhancement of images containing snow
EP2551794A2 (en) * 2011-07-28 2013-01-30 Hitachi Ltd. Onboard environment recognition system
US8582809B2 (en) 2010-06-28 2013-11-12 Robert Bosch Gmbh Method and device for detecting an interfering object in a camera image
US20140168440A1 (en) * 2011-09-12 2014-06-19 Nissan Motor Co., Ltd. Three-dimensional object detection device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102011122457A1 (en) * 2011-12-24 2013-06-27 Connaught Electronics Ltd. Method for operating a camera arrangement, camera arrangement and driver assistance system
US9304081B2 (en) * 2013-10-24 2016-04-05 The Regents Of The University Of Michigan Ice and water detection system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009027089A2 (en) * 2007-08-30 2009-03-05 Valeo Schalter Und Sensoren Gmbh Method and system for weather condition detection with image-based road characterization
US20100296700A1 (en) * 2007-10-31 2010-11-25 Adc Automotive Distance Control Systems Gmbh Method and device for detecting the course of a traffic lane
US20100121577A1 (en) * 2008-04-24 2010-05-13 Gm Global Technology Operations, Inc. Three-dimensional lidar-based clear path detection
US8285059B2 (en) 2008-05-20 2012-10-09 Xerox Corporation Method for automatic enhancement of images containing snow
US8582809B2 (en) 2010-06-28 2013-11-12 Robert Bosch Gmbh Method and device for detecting an interfering object in a camera image
EP2551794A2 (en) * 2011-07-28 2013-01-30 Hitachi Ltd. Onboard environment recognition system
US20140168440A1 (en) * 2011-09-12 2014-06-19 Nissan Motor Co., Ltd. Three-dimensional object detection device

Also Published As

Publication number Publication date
DE102014110663A1 (en) 2016-02-04

Similar Documents

Publication Publication Date Title
CN104512411B (en) Vehicle control system and imaging sensor
KR102555362B1 (en) Shutterless Far Infrared (FIR) Cameras for Automotive Safety and Driving Systems
JP2022111151A (en) Imaging apparatus and control method of imaging apparatus
CN109565546B (en) Image processing apparatus, information generating apparatus, and information generating method
JP7310869B2 (en) Solid-state imaging device and electronic equipment
US10511789B2 (en) Infrared imaging device, control method thereof, and vehicle
JP7225600B2 (en) Information processing device, information processing method, program
JP6354578B2 (en) Imaging system
US9398227B2 (en) System and method for estimating daytime visibility
US10704957B2 (en) Imaging device and imaging method
KR20170010315A (en) Systems and methods for haziness detection
US10957028B2 (en) Image generation device, image generation method and recording medium
US11330202B2 (en) Solid-state image sensor, imaging device, and method of controlling solid-state image sensor
EP2533008A1 (en) Vehicle optical sensor system
JP6971933B2 (en) Image processing equipment and imaging equipment
WO2019073726A1 (en) Solid-state imaging element, method for driving solid-state imaging element, and electronic device
US11089230B2 (en) Capturing apparatus, capturing module, capturing system, and capturing apparatus control method
IL294199A (en) Microlensing for real-time sensing of stray light
US10452076B2 (en) Vehicle vision system with adjustable computation and data compression
WO2016015965A1 (en) Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle
US20190342482A1 (en) Night vision apparatus
US10873732B2 (en) Imaging device, imaging system, and method of controlling imaging device
KR102374016B1 (en) Image processing apparatus, image processing method, and imaging apparatus
WO2023100640A1 (en) Semiconductor device, signal processing method, and program
KR102583256B1 (en) Method and system for providing confidence estimates to depth maps

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15741515

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15741515

Country of ref document: EP

Kind code of ref document: A1