WO2005018231A1

WO2005018231A1 - A method for adjust the bandwidth of bit stream

Info

Publication number: WO2005018231A1
Application number: PCT/CN2004/000910
Authority: WO
Inventors: Jing Wang; Xiaorong Wang; Ge Guo; Xiaoxia Wei
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2003-08-13
Filing date: 2004-08-06
Publication date: 2005-02-24
Also published as: CN100388780C; CN1581970A

Abstract

This invent is related to a method for adjust the bandwidth of bit stream. Firstly, analyze and calculate the bandwidth features of each endpoint, and then according to said analysis separately calculate the adjustment of each endpoint. Finally, depending on the calculated result, adjust the bandwidth of each endpoint correspondingly. Applying this invention of a method for adjust the bandwidth of bit stream, it can avoid occupying mass resource of MCU, and decreasing the complex and cost of MCU, and be in favor of extending multi-picture capacity.

Description

Method for bandwidth equalization of stream

The present invention relates to the field of multimedia communications, and in particular, to a bandwidth equalization method for a code stream. Background technique

At present, in video conference systems, terminals can transmit multi-screen videos with different contents to each other. In the process of multi-screen transmission, each terminal can only enjoy the corresponding bandwidth according to a preset ratio, and cannot transmit multi-screen according to the transmission. The complexity of the video dynamically changes its occupied bandwidth. Therefore, during the multi-view video transmission process, since the complexity of the multi-view video changes at any time, and the bandwidth occupied by each terminal will not change accordingly, thereby causing the bandwidth shortage of some terminals and the idle bandwidth of other terminals, greatly reducing the bandwidth. The overall bandwidth utilization of the video conference system.

To solve the above problems, there are two main ways to implement the multi-picture implementation in the existing video conference system:

(1) A multi-picture implementation scheme for secondary encoding and decoding, refer to FIG. 1:

The main process of the secondary encoding and decoding multi-picture implementation scheme is as follows: First, a multipoint control unit (MCU, Multipoint Control Unit) in the video conference system decodes the video code streams from each terminal separately, and obtains corresponding multi-channel solutions Image

The MCU then extracts the solved images and scales them to the corresponding proportions and combines them; Finally, the combined images are overall encoded, and the encoded single video stream is fed back to each terminal.

By applying the secondary encoding and decoding scheme, each terminal can use the secondary encoding and decoding for the purpose of dynamically adjusting its own bandwidth for multi-picture images of different complexity, thereby achieving the effect of reasonable bandwidth allocation and balance.

However, the above-mentioned method for achieving bandwidth equalization based on the secondary encoding and decoding scheme has the following disadvantages:

Achieving the equalization of bandwidth between terminals requires secondary encoding and decoding processing, which requires the video conference system to have the ability to carry complex operations, and also requires high hardware implementation costs;

Since the implementation process of the secondary encoding and decoding is centralized processing on the MCU side, it will occupy a large amount of operating resources of the MCU, which is not conducive to the expansion of multi-picture capacity.

(2) A multi-picture implementation scheme for a composite code stream, as shown in FIG. 2:

The main process of the multi-picture implementation scheme of the composite code stream is as follows: each terminal in the video conference system occupies the corresponding bandwidth capacity according to the proportion of the sub-pictures transmitted in the multi-pictures;

Each terminal encodes according to the size of the bandwidth capacity it occupies, and sends the encoded video stream to the MCU separately;

The MCU extracts the video stream sent from each terminal separately, and records the corresponding bandwidth capacity occupied by each terminal according to the video stream extracted from each terminal, and then synthesizes the extracted video streams in order. ; The single-channel video stream synthesized by MCl ^ Bar is then fed back to each terminal with the corresponding bandwidth capacity occupied by each terminal, and each terminal performs a decoding operation.

However, in the above-mentioned multi-picture implementation scheme based on composite code stream, there is no proposed method for controlling bandwidth equalization among various terminals, so that the overall bandwidth utilization rate of the video conference system adopting the multi-picture scheme of composite code stream Cannot be effectively improved. Summary of the invention

The present invention proposes a method for bandwidth equalization of a code stream, so as to solve the problem that in the prior art, it can not simultaneously meet the requirements of saving MCU resources and improving overall bandwidth utilization.

To solve the above problem, the present invention proposes a method for bandwidth equalization of a code stream, which includes the following steps:

(1) Analyze and calculate the bandwidth characteristics of each terminal;

(2) According to the analysis and calculation result of step (1), separately calculate the adjustment amount of the bandwidth balance of each terminal;

(3) According to the adjustment amount calculated in step (2), adjust each terminal's own bandwidth accordingly.

Wherein, in step (1), the analysis and calculation of the bandwidth characteristics of each terminal are processed by each terminal; in step (2), each terminal's bandwidth balance adjustment is calculated by each terminal. ; In step (3), each terminal's own bandwidth is adjusted accordingly by each terminal.

The step (1) specifically includes:

(11) Each terminal calculates an average quantization step size value of each terminal in each cycle time;

(12) Each terminal calculates its own bandwidth utilization rate within the period of each cycle; (13) Each terminal calculates the overall bandwidth redundancy ratio in the period of each cycle. The step (11) further includes:

(111) Each terminal decodes each macroblock in the video image separately to obtain a quantization step size value of each macroblock;

(112) After adding the quantization step size values of all macroblocks, divide by the total number of macroblocks contained in the video image to obtain the average quantization step size value.

Wherein, the bandwidth utilization rate in the step (12) is obtained by the following formula: Bandwidth utilization rate = equal to the actual number of coded bits of the terminal within each period of time / the target number of bits of the terminal within each period of time.

The overall bandwidth redundancy ratio in the step (13) is obtained by the following formula: Overall bandwidth redundancy ratio = (target total number of bits per cycle time-actual number of decoded bits per cycle time) / per cycle time Target total number of bits

The target total number of bits is equal to the sum of the target bits of all terminals; the actual total number of decoded bits is equal to the sum of the actual number of decoded bits of all terminals.

The step (2) specifically includes: if a terminal meets

(al) the terminal has the largest average quantization step size among all terminals;

(a2) The bandwidth utilization rate of the terminal> = M%;

3) The overall bandwidth redundancy rate> = N%;

Then the terminal performs bandwidth increase adjustment, the adjustment amount = the initial target bandwidth of the terminal X

P%; M is a preset bandwidth utilization threshold value, N is a preset first overall bandwidth redundancy ratio threshold value, and P is a preset adjustment amplitude threshold value.

The step (2) specifically includes: within a period of time, if a terminal satisfies

(bl) overall bandwidth redundancy rate <=%;

(b2) the current target bandwidth of the terminal> the initial target bandwidth; Then the terminal performs a bandwidth reduction adjustment. The adjustment amount = the initial target bandwidth XP% of the terminal; the K is a preset second overall bandwidth redundancy ratio threshold, and P is a preset adjustment amplitude threshold.

The step (2) further specifically includes: if a terminal satisfies the current target bandwidth / initial target bandwidth <Q% within a cycle time, the terminal performs bandwidth increase adjustment, and the adjustment amount = initial target bandwidth-current target bandwidth The Q is a preset current bandwidth utilization threshold.

The invention improves the overall bandwidth utilization rate; and realizes the bandwidth equalization in each terminal, thereby decomposing the process that was originally processed by the MCU into a process that is distributed and processed by multiple terminals, avoiding occupying a large amount of operating resources of the MCU, and reducing It reduces the computational complexity and hardware implementation cost of the MCU. At the same time, it is beneficial for the MCU to expand the multi-picture capacity and improve the overall quality of the multi-picture video. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a system block diagram of a multi-picture implementation scheme using a secondary codec in the prior art; FIG. 2 is a system block diagram of a multi-picture implementation scheme using a composite code stream in the prior art; FIG. 3 is a block diagram of a code stream of the present invention The main flow chart of the bandwidth equalization method. DETAILED DESCRIPTION OF THE INVENTION The bandwidth equalization method of a code stream of the present invention is designed based on the foregoing principles. In the video conference system, when a multi-picture implementation scheme based on a composite code stream is used to transmit a multi-picture video code stream for a conference, each terminal participating in the conference will receive each sub-picture in the multi-picture. In this way, each terminal can accurately determine the current bandwidth demand of each terminal by analyzing the video stream bandwidth characteristics of each terminal (including itself); then each terminal can separately determine the current bandwidth demand of each terminal at the conference. Adjust its own bitstream bandwidth accordingly; Thus, in the video conference system, each terminal completes the overall bandwidth occupation of the system, so that each terminal can dynamically adjust its own occupied bandwidth according to the complexity of the received sub-pictures, thereby improving the overall system bandwidth utilization.

The bandwidth equalization method of the code stream of the present invention is designed based on the above principles. The core purpose of this solution is to decompose the bandwidth of each terminal by the MCU in the prior art into a balanced distribution of multiple terminals to overcome the concentration. Disadvantages of occupying MCU operating resources during processing. The implementation scheme of the present invention will be described in detail below with reference to the drawings.

First set some parameter thresholds in this solution: Set the period threshold T = 10ms; the bandwidth utilization threshold M = 80; the first overall bandwidth redundancy ratio threshold N = 50; the second overall bandwidth redundancy ratio threshold K = 20; adjustment Amplitude threshold P = 10; Bandwidth multiplication threshold L = 1.5; Current bandwidth utilization threshold Q = 50. The thresholds set in the above can be flexibly set according to the needs of the system, and are not limited to the values listed here.

Referring to FIG. 3, in step 10, each terminal participating in the video conference separately analyzes and calculates the video stream bandwidth characteristics of the participating terminals, that is, in addition to analyzing and calculating its own video stream bandwidth characteristics, each terminal must also receive The video stream of each other terminal analyzes and calculates the bandwidth characteristics of the video stream of each other terminal (excluding itself).

Each terminal analysis and calculation of the video stream bandwidth characteristics of each terminal includes: Calculating the average quantization step size value of each terminal in each cycle time. The quantization step size value refers to an average quantization step size in a cycle time closest to the detection point. Because when each terminal sends a video image, the entire video image is divided into several macroblocks, each macroblock has a unique quantization step size QP, and then each macroblock is encoded separately; therefore, each terminal When receiving a video image, when decoding the video image, the image must also be decoded separately. The QP value of each macroblock in the image, where the average quantization step size value of each terminal is obtained by the following formula:

Average quantization step size of each terminal = (QP, + QP ₂ + QP ₃ + ..... + QP _n ) / n where (5 ?, is the quantization step size of the first macroblock of the image, QP ₂ is the quantization step size of the second macroblock of the image, ..... (^ „is the quantization step size of the nth macroblock of the image, where n is the total number of divided macroblocks in the entire image Each terminal shall calculate its own average quantization step size value and the average quantization step size value of each other terminal in each cycle time, and then compare the average quantization step size values.

Calculate the bandwidth utilization of each terminal within each cycle time. For each terminal, the bandwidth utilization is obtained by the following formula:

Bandwidth utilization = actual number of coded bits of the terminal per cycle time / target number of bits of the terminal within each cycle time

Here, the actual number of coded bits of the terminal refers to the number of code stream bits actually coded by the terminal encoder in each cycle time; the target target number of bits refers to the number of code stream bits allowed by the terminal encoder in each cycle time.

Calculate the overall system bandwidth redundancy ratio. For each terminal, the overall bandwidth redundancy ratio is obtained by the following formula:

Overall bandwidth redundancy rate = (target total number of bits per cycle time-actual number of decoded bits per cycle time) / target total number of bits per cycle time

The target total number of bits is equal to the sum of the target bits of all terminals; the actual decoded total number of bits is equal to the sum of the actual decoded bits of all terminals; the actual number of decoded bits of the terminal is Refers to the number of bits actually decoded by the terminal decoder during each cycle time; the overall bandwidth redundancy ratio result calculated by all terminals in the system at the same time is the same.

In step 20, each terminal should refer to the bandwidth characteristic parameters that have been analyzed and calculated in step 10 to calculate its corresponding bandwidth adjustment amount. The bandwidth adjustment amount is divided into two cases: increasing adjustment mode and decreasing adjustment mode.

Bandwidth increase adjustment mode: In a certain period of time, when a terminal obtains the bandwidth characteristics through the corresponding analysis and calculation in step 10, it simultaneously meets the following three conditions:

The terminal compares its average quantization step size value with the average quantization step size value of all other terminals one by one to obtain the maximum average quantization step size value;

The terminal calculates its own bandwidth utilization rate> = 80% (where 80 is the preset bandwidth utilization threshold M described above);

The overall bandwidth redundancy ratio of the system> = 50% (where 50 is the preset first overall bandwidth redundancy ratio threshold N);

Then the terminal enters the bandwidth increase adjustment mode, and the terminal starts to calculate the adjustment amount of the current increase adjustment, and the adjustment amount = the terminal's initial target bandwidth X 10% (where 10 is the preset adjustment threshold P above) .

However, for any terminal, after entering the increase mode adjustment, the increase adjustment is not unlimited. It can only increase up to 1.5 times the initial target bandwidth of the terminal (where 1.5 is the preset bandwidth described above). Multiplication threshold L). The initial target bandwidth is determined by three factors: the overall system bandwidth, the multi-picture format, and the proportion of terminal sub-pictures in the multi-picture. There is also a special processing method for adjusting the bandwidth increase mode. When a certain terminal calculates the bandwidth characteristics through the corresponding analysis and calculation in step 10, the current target bandwidth / initial target bandwidth is <50% (of which, Here, 50 is the preset current bandwidth utilization threshold Q described above. In order to accelerate the bandwidth adjustment speed of the terminal, the terminal may be forced to recover to the original target bandwidth at one time, that is, at this time, the terminal is increased. Mode adjustment amount = initial target bandwidth-current target bandwidth. The current target bandwidth refers to the currently programmed bandwidth of the terminal encoder.

Bandwidth reduction adjustment mode: In a certain period of time, when a terminal obtains the bandwidth characteristics through the corresponding analysis and calculation in step 10, the following two conditions are satisfied at the same time: The overall system bandwidth redundancy ratio <= 20% (of which 20 Is the preset second bandwidth utilization threshold K); the current target bandwidth of the terminal> the initial target bandwidth of the terminal;

Then the terminal enters the bandwidth reduction adjustment mode, and the terminal starts to calculate the adjustment amount for the current reduction adjustment, where the adjustment amount = the terminal's initial target bandwidth X 10% (here 10 is the preset preset adjustment range alarm value P ). The bandwidth reduction adjustment has a limit, and the minimum terminal bandwidth can only be reduced to the initial target bandwidth of the terminal.

Then, each terminal adjusts accordingly to increase or decrease according to the adjustment amount calculated by itself, so as to achieve the bandwidth balance among the terminals and improve the overall system bandwidth utilization. The bandwidth adjustment of each terminal can adjust the size of its own bandwidth by setting instructions in advance for each terminal and automatically responding to instructions.

Each of the above terminals simultaneously analyzes and calculates the bandwidth characteristics of each terminal at the same cycle time, and then obtains the respective bandwidth adjustment amount within each cycle time, and then each terminal performs corresponding bandwidth adjustment within the cycle time. The cycle time is determined by the overall bandwidth of the video conference system application. During the process of the video conference, each terminal All the terminals perform cyclic analysis and calculation of the bandwidth characteristics of each terminal, and then obtain the adjustment amount of each bandwidth within each cycle time, and then adjust accordingly; this achieves the purpose of each terminal dynamically adjusting its own bandwidth in real time. Analysis and calculation are performed; each terminal calculates and determines its own bandwidth balance adjustment amount according to the result of the analysis calculation; and finally, each terminal adjusts its own bandwidth according to the adjustment amount. In fact, the above operations do not have to be performed by the terminal itself. Alternatively, a test device can be installed outside each terminal, and the external test device can complete the analysis and calculation of the bandwidth characteristics of each terminal. As a result, the bandwidth balance adjustment amount of each terminal is calculated and determined; finally, the bandwidth of each terminal is adjusted correspondingly according to the adjustment amount, and the implementation thereof is the same as that performed by each terminal itself, which is not described herein again.

The above are only preferred embodiments of the present invention. It should be noted that, for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made. These improvements and retouches should also be made. It is regarded as the protection scope of the present invention.

Claims

Rights request

1. A method for bandwidth equalization of a code stream, which is characterized by including the following steps:

(1) Analyze and calculate the bandwidth characteristics of each terminal separately;

(2) According to the analysis and calculation result of step (1), calculate the adjustment amount of the bandwidth balance of each terminal separately;

2. The method for bandwidth equalization of a code stream according to claim 1, characterized in that, in step (1), the analysis and calculation of the bandwidth characteristics of each terminal are separately processed by each terminal; in step (2) Each terminal calculates the bandwidth balance adjustment amount separately, which is handled by each terminal. In step (3), each terminal's own bandwidth is adjusted by each terminal.

3. The method for bandwidth equalization of a code stream according to claim 2, wherein the step (1) specifically comprises:

(12) Each terminal calculates its own bandwidth utilization rate within the period of each cycle; (13) Each terminal calculates the overall bandwidth redundancy rate during the period of each cycle.

4. The method for bandwidth equalization of a code stream according to claim 3, wherein the step (11) further comprises:

(111) Each terminal decodes each macroblock in the video image separately to obtain a quantization step size value of each macroblock; (112) After adding the quantization step size values of all macro blocks, divide by the total number of macro blocks contained in the video image to obtain the average quantization step size value.

5. The bandwidth equalization method for a code stream according to claim 3, wherein the bandwidth utilization rate in the step (12) is obtained by the following formula:

Bandwidth utilization rate = equal to the actual number of coded bits of the terminal per cycle time / target number of bits of the terminal within each cycle time.

6. The method for bandwidth equalization of a code stream according to claim 3, wherein the overall bandwidth redundancy ratio in the step (13) is obtained by the following formula:

Overall bandwidth redundancy ratio = (target total number of bits per cycle time-actual total number of decoded bits per cycle time) / target total number of bits per cycle time;

The target total number of bits is equal to the sum of the target bits of all terminals; the actual decoded total number of bits is equal to the sum of the actual decoded bits of all terminals.

7. The method for bandwidth equalization of a code stream according to claim 3, wherein the step (2) specifically comprises: within a cycle time, if a terminal meets

(a2) The bandwidth utilization rate of the terminal> = M%;

(a3) overall bandwidth redundancy ratio == N%;

Then, the terminal performs bandwidth increase adjustment, the adjustment amount = the terminal's initial target bandwidth XP%; M is a preset bandwidth utilization threshold, N is a preset first overall bandwidth redundancy ratio threshold, P It is a preset adjustment amplitude threshold. 8. The bandwidth equalization method for a code stream according to claim 7, wherein the upper limit of the bandwidth increase adjustment of the terminal is L times the initial target bandwidth of the terminal; and L is a preset bandwidth multiplication Threshold.

9. The method for bandwidth equalization of a code stream according to claim 3, wherein the step (2) specifically comprises: within a cycle time, if a terminal simultaneously meets

(bl) overall bandwidth redundancy rate <=%;

(i) the current target bandwidth of the terminal> the initial target bandwidth;

Then, the terminal performs bandwidth reduction adjustment, the adjustment amount = the terminal's initial target bandwidth X P%; K is a preset second overall bandwidth redundancy ratio threshold, and P is a preset adjustment width threshold.

10. The method for bandwidth equalization of a code stream according to claim 3, wherein the step (2) specifically comprises: within a period of time, if a terminal meets a current target bandwidth / initial target bandwidth <Q%, Then, the terminal performs bandwidth increase adjustment, and the adjustment amount = initial target bandwidth-current target bandwidth; the Q is a preset current bandwidth utilization threshold.

11. The bandwidth equalization method for a code stream according to claim 3, 5, 6, 7, 9, or 10, wherein the cycle time is a preset cycle threshold.