DE10007579A1 - Method and device for coding a video signal and a speech signal, computer-readable storage medium and computer program element - Google Patents

Abstract

The audio signal is encoded in a first channel and the video signal in a second channel. It is determined when no audio signal is encoded and during this time the video signal is encoded with higher quality.

Description

The invention relates to a method and an apparatus for coding a video signal and a speech signal, as well as a computer readable storage medium and a Computer program element.

Such a method and such a device are known for example from [1], [2], [3] or [4]. In such a known video communication system 200 , an audio signal 201 is encoded to an encoded audio signal 203 by means of an audio encoder 202 (cf. FIG. 2). The coded audio signal 203 exclusively has signal components in a first frequency range, which is referred to as the first channel 204 in the following.

A video signal 205 is fed to a video encoder 206 and encoded there to form an encoded video signal 207 . The encoded video signal 207 has signal components in a second frequency range that differs from the first frequency range, that is to say the first channel 204 . The second frequency range is referred to below as second channel 208 .

The coded audio signal 203 and the coded video signal 207 are both fed to a multiplexer 209 and transmitted by the multiplexer 209 as a coded overall signal 210 from the transmitter 211 via a physical channel 212 to a receiver 213 .

A demultiplexer 214 is contained in the receiver 213 , with which the received encoded total signal 210 is again divided into a received encoded audio signal 215 and a received encoded video signal 216 . The received encoded audio signal 215 has signal components in the first channel and the received encoded video signal has signal components in the second channel.

The received coded audio signal is fed to an audio decoder 217 and decoded by this to a reconstructed audio signal 218 .

The received coded video signal 216 is fed to a video decoder 219 and decoded by the latter into a reconstructed video signal 220 .

In the context of video telephony, communication in the video communication system 200 is bidirectional, that is to say in both directions, so that a receiver is also included on the transmitter side and a transmitter is also included on the receiver side.

As can be seen from FIG. 2, the coding of the audio signal 201 and the video signal 205 takes place separately from one another in two different channels. A disadvantage of this method and such a device is, in particular, that the first frequency range, that is to say the first channel, remains reserved, that is to say is not available for the transmission of further data, although no audio signals, that is to say, for example, no voice signals, are available via the first channel must be transferred.

An audio signal 201 is to be understood in particular as a voice signal. The coding of a speech signal is known, for example, from [5]: [5] also describes the determination of speech pauses and the coding of speech pauses.

According to the video encoding described in [6], [7], [4] Standards is used in the coding of each  Video images the resolution, that is, for example Coding the video images used quantization of the individual DCT coefficients according to the Discrete Cosine Transformation (DCT) have been transformed depending on the fill level of the output buffer of the video encoder regulated.

A disadvantage of this approach is that to see that some response time regulating the quantization used cannot be avoided, that is, that, for example, an increased resolution within the Quantization would be possible because an increased bandwidth for Transfer of data is available, though through the level in the output buffer is not yet visible is.

In the known systems is only a regulation of data rate used to encode the video signal in the form an iterative adjustment of the quality or the data rate possible.

Thus, the invention addresses the problem of coding a video signal and an audio signal as part of a Video communication system the entire available Make better use of bandwidth.

The problem is solved by the method and the device for Encoding a video signal and a speech signal which computer-readable storage medium and the computer program Element with the characteristics according to the independent Claims resolved.

In a method of encoding a video signal and one Audio signal becomes the audio signal in a first channel coded. The video signal is encoded in a second channel. In other words, this means that the audio signal is in one encoded audio signal is encoded, the signal components in  has a first frequency range and that Video signal is encoded into a coded video signal, which signal components in a second frequency range, which is completely different from the first frequency range differs. While coding the Video signal and audio signal is determined when none Audio signal is encoded. During the time when no Audio signal is encoded, the video signal is higher Quality encoded or a larger number of pictures coded.

In a device for coding a video signal and an audio signal, a processor is provided which is set up in such a way that the following method steps can be carried out:

• - The audio signal is encoded in a first channel,
• The video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded, and
• - during the time in which no audio signal is encoded,

the video signal is encoded with higher quality.

Furthermore, a computer-readable storage medium is provided, in which a computer program for coding a video signal and an audio signal is stored, which, when executed by a processor, has the following method steps:

• - The audio signal is encoded in a first channel,
• The video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded and
• - during the time in which no audio signal is encoded,

the video signal is encoded with higher quality.

In the case of a computer program element for encoding a video signal and an audio signal, the following method steps are carried out when the computer program element is executed by a processor:

• - The audio signal is encoded in a first channel,  
• The video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded, and
• - during the time in which no audio signal is encoded,

the video signal is encoded with higher quality.

With the invention it is now possible, in addition, automatically more data rate for video coding To make available.

It was recognized according to the invention that in the time while one communicator is listening to another, that is, it does not itself generate speech signals, even that Movement of the listener is less, which also causes the Video signals, in particular through movement of the individual Objects contained in the picture, including the recorded one Communication participant, are generated significantly be reduced.

So, for example, because of the fact that a communication participant has no voice signals automatically generates the quality of the encoded video signal increase without examining the buffer level generated time delay.

In this way, a very simple, inventive Criterion for regulating the quality of the encoded generated Video signal found.

Another advantage of the invention is that even before encoding a digitized picture or of a digitized difference image standing bandwidth can be estimated well, what with the The prior art method is not possible because these are based on information for coding an image during coding based on the buffer level of the Output buffer (i.e. only after coding) as Use rule criterion.  

Preferred developments of the invention result from the dependent claims.

The configurations described below relate both on the process, the device the computer readable Storage medium as well as on the computer program element.

The invention can be implemented both as a computer program as well as by means of a special, coordinated Hardware, for example using a special electronic circuit.

During the time that no audio signal is being encoded, the video signal in the first channel and in the second channel be encoded.

This way, during the break, in which no Audio signal is generated, which is actually not for the Video encoding available bandwidth used to the quality of the video signal to be encoded further increase. This way the whole is available standing bandwidth used much better and a achieved significant improvement in image quality.

The audio signal can be a voice signal.

In a further embodiment of the invention, it is provided that the video signal digitized images and digitized difference images, for example digitized images and digitized difference images that are encoded according to video coding standards as they are e.g. B. are described in [6], [7], [4].

The additional through the increased quality of the video signal encoded part of the video signal is preferred in the first Channel coded.  

The quality of the video signal to be encoded can for example, be increased by the quantization in the context of coding the video signal to be coded is increased, that is, as part of the quantization of the transformation coefficients to be coded, for example DCT coefficients with greater accuracy, that is with a larger number of quantization levels.

Alternatively, the quality of the video signal to be encoded by increasing the number of digitized Difference images between two digitized images be fully encoded, reducing the number the images remain the same per unit of time. In this way will be errors in difference images, which can be found in a large Number of consecutive difference images accumulate, reducing, thereby reducing the quality of the Total video stream is increased.

In a further embodiment, the quality of the encoding video signal increased in that at least one Part of an additional digitized image is encoded. In this context, several additional ones can also be used Inter pictures are encoded.

According to this configuration, either a part, for example a block, a macroblock or a certain, suitably chosen area of the image not as Differential picture (inter picture) coded, but as a so-called Intra picture.

An embodiment of the invention is in the figures shown and explained in more detail below:

Show it  

Fig. 1 is a sketch of a video communication system according to an embodiment of the invention;

Fig. 2 is a sketch of a video communication system according to the prior art;

Fig. 3 is a sketch of the video encoder according to the embodiment of the invention.

Fig. 4 is a sketch of the video decoder according to an embodiment of the invention.

Fig. 1 shows a video communication system 100 for encoding, transmitting and decoding an audio signal 101 and a video signal 105.

The audio signal 101 is fed to an audio encoder 102 , as a result of which the audio signal 101 is encoded to a coded audio signal 103 . The encoded audio signal 103 has signal components in a first frequency range, that is to say in a first channel 104 .

According to this embodiment, the audio signal is on Voice signal.

The video signal 105 is fed to a video encoder 106 and encoded by it to form an encoded video signal 107 .

A speech pause determination unit 108 is also provided, to which the audio signal, that is to say the speech signal 101 , is supplied. In the speech pause determination unit 108 , time intervals are determined in which no audio signal, that is to say no voice signal, has to be encoded.

The information about the determined speech pause is fed as a speech pause signal 109 to a video encoder control 110 .

In the speech pause determination unit, the energy of the speech signal is determined, and the speech pause signal is set to a first logic binary value (logic "1") if the energy of the speech signal is below a predetermined threshold, that is to say if it is assumed that no speech signal is present on the Audio encoder 102 is present to be encoded.

However, if the energy of the speech signal is above the threshold value, the speech pause signal 109 is set to a second logic binary value (logic "0"), which indicates that a speech signal is encoded by the audio encoder 102 and thus that for the first channel 104 available bandwidth is required for the speech signal.

If the speech pause signal 109 is at the first logical binary value, the video encoder controller 110 supplies a control signal 111 to the video encoder 106 .

The regulation of the video encoding is explained in more detail below in connection with FIG. 3.

If the speech pause signal 109 has the first logical binary value, then the video encoder 106 is regulated in such a way that the coded video signal also has signal components in the first frequency range, that is to say in the first channel 104 .

Furthermore, an output buffer fill level signal 113 is fed as an output signal of the video encoder 106 to a further input of the video encoder 106 .

Thus, in the context of this embodiment too additionally a buffer fill level control, as in the state the technology is provided, realized.

The coded audio signal 103 and the coded video signal 107 are fed to a multiplexer 114 , where they are multiplexed to form a coded overall signal 115 .

The encoded overall signal is transmitted from the transmitter 116 to a receiver 118 via a physical channel 117 , in this exemplary embodiment a telephone connection.

A demultiplexer 119 is provided in the receiver 118 , by means of which the encoded total signal 115 is divided into a received encoded audio signal 120 and a received encoded video signal 121 .

The video signal is processed according to the MPEG4- Coded as standard.

The coded video signal thus contains the information that the coded signal is a video signal. This is recognized in the demultiplexer 119 and the video signal, which has signal components in the first frequency range, that is to say in the first channel, is nevertheless fed to the video decoder 122 . The received coded video signal 107 fed to the video decoder 122 is decoded into a reconstructed video signal 123 .

The received coded audio signal also contained in the coded overall signal 115 is fed to an audio decoder 124 , by which it is decoded into a reconstructed audio signal 125 .

Fig. 3106 shows the structure of the video encoder in detail.

The video signal 105 to be coded is fed to a switch unit 301 via an input 300 of the video encoder 106 .

The video signal 105 to be coded has digitized coding information, that is to say, for example, brightness information and / or color information, each of which is assigned to a pixel of a digitized image.

Depending on whether an intra-picture or a so-called inter-picture is to be coded, either the coding information resulting from the video signal to be coded minus the video signal of a temporally preceding picture 302 is fed to a transformation unit 303 via the switch unit 301 or the entire one Coding information 302 of the digitized picture to be coded without forming a difference with a temporally preceding picture in the case of intra-picture coding.

In the transtormation unit 303 , the coding information or differential coding information supplied to it is coded to DCT coefficients 304 by means of a discrete cosine transformation (DCT). The DCT coefficients 304 are fed to a quantization unit 305 , where they are quantized according to a freely selectable resolution (quantization level), which also determines the number of quantized DCT coefficients 306 to be processed.

The quantized DCT coefficients 306 are subjected to length coding in a run-length coding unit 307 and then the coded coefficients 308 are written into an output buffer 309 , from which they are read out again and fed to the multiplexer 114 .

Furthermore, the quantized DCT coefficients 306 are fed in a feedback loop to an inverse quantization unit 310 , there inversely quantized to inversely quantized DCT coefficients 311, and after an inverse transformation unit 312 has been supplied, inversely quantized DCT coefficients 313 are written into a memory 314 .

A motion estimation is carried out in a unit for motion estimation 315 and determined motion vectors 316 are fed to the RLC unit 307 .

If the speech pause signal 109 has the first logical binary value, then the video control signal 111 , which is applied to the quantization unit 304 , increases the resolution used for quantization, that is to say the number of quantization stages used.

The additional one created by the increased quantization Data stream is encoded in the first channel, i.e. the corresponding signal components are in the first Frequency range transformed.

After demultiplexing, the received coded video signal 121 is fed to an RLD unit 401 (run-length decoding) via an input 400 of the video decoder 123 (cf. FIG. 4).

After run-length decoding has taken place, the run-length decoded quantized DCT coefficients 402 are fed to an inverse quantization unit 403 . The determined run-length decoded motion vectors 404 are fed to a unit for motion compensation 405 .

The inversely quantized DCT coefficients 406 generated by inverse quantization carried out in the inverse quantization unit 403 are fed to an inverse transformation unit 407 .

In the inverse transformation unit 407 , an inverse discrete cosine transformation is applied to the inversely quantized DCT coefficients 406 , whereby inversely transformed DCT coefficients 408 are generated. The inversely transformed DCT coefficients 408 are stored in a memory 409 .

In the case of a difference image, the reconstructed video signal 125 is formed by means of the motion compensation unit 405 using the run-length decoded motion vectors 404 , the inversely transformed DCT coefficients being added to the image formed by means of motion compensation. If it is a so-called intra-image, the reconstructed video signal 125 is formed directly from the memory 409 by reading out the inversely transformed DCT coefficients 408 .

The following are some alternatives to the one above illustrated embodiment:

The speech pauses can be determined directly by the audio encoder and, for example, by means of the coding according to the standard for coding a speech signal described in [5], a coding of a speech pause can be supplied directly to the video encoder controller 110 as a speech pause signal 109 .

Furthermore, it is not absolutely necessary that the Improve the quality of the video signal to be encoded by increasing the resolution in the quantization unit done, it can also be increased by transferring it Number of DCT coefficients take place or by Decrease in the number of inter-pictures, that is, from Difference images between two transmitted intra images constant number of images per unit of time and it additional INTER images can be transmitted. It can  even part of an intra-image or an entire intra- Image coded and transmitted during the pause.

It also does not require the video signal Signal components in the first frequency range. It is also possible according to the invention, only To encode signal components in the second frequency range. In this case the criterion of a language break is considered immediate control criterion used, whereby is exploited that a listener is usually only very little moved and therefore very little Motion information is to be coded.  

The following publications are cited in this document:
[1] ITU standard H.223, Multiplexing Protocol for Low Bit Rate Multimedia Communication, 1996
[2] ITU Standard H. 320, Narrow-Band Visual Telephone Systems and Terminal Equipment, 1996
[3] ITU standard H. 323, Packet-Based Multimedia Communication Systems, 1996
[4] ISO 14496, MPEG4 communication standard 1998
[5] ITU standard G.723.1, Dual Rate Speech Coder for Multimedia Communication Transmissions at 5.3 and 6.4 K-Bit / S, 1986
[6] ITU standard H.261, video codec for audio-visual services at P × 64 K-Bit / S. 1993
[7] ITU Standard H.263, Video Codec for Very Low Bit-Rate Multimedia Communication, 1996

Claims (11)

1. Method for coding a video signal and an audio signal,

• in which the audio signal is encoded in a first channel,
• in which the video signal is encoded in a second channel,
• - in which it is determined when no audio signal is encoded, and
• - In which during the time in which no audio signal is encoded, the video signal is encoded with higher quality.

2. The method according to claim 1, at which during the time when no audio signal is encoded the video signal in the first channel and in the second Channel is encoded. 3. The method according to claim 1 or 2, where the video signal digitized images and digitized difference images. 4. The method according to any one of claims 1 to 3, where the through the increased quality of the video signal additional encoded part of the video signal in the first Channel is encoded. 5. The method according to any one of claims 1 to 4, which increases the quality of the video signal to be encoded is by quantization as part of the coding of the encoding video signal is increased. 6. The method according to any one of claims 3 to 5, which increases the quality of the video signal to be encoded is by the number of digitized difference images is reduced between two digitized images or by transmitting additional INTER images.   7. The method according to any one of claims 3 to 6, which increases the quality of the video signal to be encoded is by adding at least part of an additional digitized image is encoded. 8. The method according to any one of claims 1 to 7, where the audio signal is a speech signal. 9. Device for coding a video signal and an audio signal, with a processor, which is set up in such a way that the following method steps can be carried out,

• the audio signal is encoded in a first channel,
• the video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded, and
• - During the time in which no audio signal is encoded, the video signal is encoded with higher quality.

10. Computer-readable storage medium in which a computer program for encoding a video signal and an audio signal is stored, which, when executed by a processor, has the following method steps:

• the audio signal is encoded in a first channel,
• the video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded, and
• - During the time in which no audio signal is encoded, the video signal is encoded with higher quality.

11. Computer program element for encoding a video signal and an audio signal, which, when executed by a processor, has the following method steps:

• the audio signal is encoded in a first channel,
• the video signal is encoded in a second channel,
• - It is determined when no audio signal is encoded, and
• - During the time in which no audio signal is encoded, the video signal is encoded with higher quality.