US20050062746A1 - Signal-processing apparatus and electronic apparatus using same - Google Patents
Signal-processing apparatus and electronic apparatus using same Download PDFInfo
- Publication number
- US20050062746A1 US20050062746A1 US10/919,238 US91923804A US2005062746A1 US 20050062746 A1 US20050062746 A1 US 20050062746A1 US 91923804 A US91923804 A US 91923804A US 2005062746 A1 US2005062746 A1 US 2005062746A1
- Authority
- US
- United States
- Prior art keywords
- data
- processing
- parallel processor
- signal
- instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/38—Concurrent instruction execution, e.g. pipeline, look ahead
- G06F9/3877—Concurrent instruction execution, e.g. pipeline, look ahead using a slave processor, e.g. coprocessor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/38—Concurrent instruction execution, e.g. pipeline, look ahead
- G06F9/3885—Concurrent instruction execution, e.g. pipeline, look ahead using a plurality of independent parallel functional units
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to a signal-processing apparatus that performs audio and image compression/decompression at high speed by use of a parallel processor and dedicated hardware, and an electronic apparatus using the same.
- the ISO International Organization for Standardization
- the ITU-T International Telecommunication Union-Telecommunication Standardization Sector
- MPEG-4 AVC Advanced Video Coding
- the MPEG-4 AVC realizes a high image compression rate by introducing new technologies such as integer conversion of 4 ⁇ 4 pixels, intra prediction at up to nine directions, seven kinds of sub-macro-block types, up to 16 motion vectors per macro-block, multi-frame reference, a de-blocking filter in the loop and arithmetic coding, and aims at a code amount compressed to 50% of the MPEG-2 that has already been put into practical use.
- Document 1 Japanese Patent Application No. H03-508269.
- the example shown in Document 1 is a parallel processor comprising a combination of a parallel data processor of the SIMD (Single Instruction Multiple Data) type in which the number of control streams is one and the number of data streams to be processed is more than one and a parallel data processor of the MIMD (Multiple Instruction Multiple Data) type in which the number of control streams and the number of data streams are both more than one.
- SIMD Single Instruction Multiple Data
- MIMD Multiple Instruction Multiple Data
- FIG. 16 which is referred from FIG. 1 of Document 1, is a block diagram illustrating a signal-processing apparatus combining a prior SIMD parallel data processor 902 and an MIMD parallel data processor 903 .
- the signal-processing apparatus comprises a system controller 901 that controls the entire processor, the SIMD parallel data processor 902 , the MIMD parallel data processor 903 , a shared memory bus 904 and a shared memory 905 .
- the system controller 901 performs execution of application programs.
- the SIMD parallel data processor 902 comprises an overall controller 910 , calculators 911 to 914 and local memories 915 to 918 respectively.
- One calculator and one local memory constitute one processor.
- the overall controller 910 executes the program, and issues the same instruction to all of the calculators 911 to 914 .
- the calculators 911 to 914 process data stored in local memories 915 to 918 respectively based on the same issued instruction.
- the MIMD parallel data processor 903 comprises an overall controller 920 , controllers 921 to 924 , calculators 925 to 928 and local memories 929 and 932 .
- One controller, one calculator and one local memory constitute one processor.
- a different program is executed by each of the controllers 921 to 924 , a different instruction is issued to each of the calculators 925 to 928 , and the data stored in each of the local memories 929 to 932 is processed.
- the overall controller 920 performs control for synchronization and monitoring of the entire MIMD parallel data processor 903 .
- the SIMD parallel data processor 902 when the object processing is simple and the data processing amount is large, the SIMD parallel data processor 902 performs processing, whereas when the object processing is complicated and the data processing amount is small, the MIMD parallel data processor 903 performs processing.
- the speed enhancing method which improves the calculation, is used by forming the most suitable calculator for the processing that is objected with the dedicated hardware.
- Document 2 Japanese Patent Application No. 2000-118434 discloses a technology that realizes a speedup of the processing by performing the variable-length encoding/decoding of the image processing with the dedicated hardware.
- FIG. 17 which is referred from FIG. 1 of Document 2, is a block diagram illustrating an image processor 1001 combining the prior SIMD parallel data processor and the dedicated hardware.
- the image processor 1001 is connected to an external video input device 1009 , a video output device 1010 and an external memory 1011 through an external video data bus 1008 .
- the image processor 1001 comprises an instruction memory 1002 , a processor 1003 , SIMD calculating means 1004 , VLC (Variable-Length Coding) processing means 1005 , an external data interface 1006 , and an internal data bus 1007 .
- the VLC processing means 1005 comprises the dedicated hardware.
- the processor 1003 performs scalar, bit manipulation and the issuance of comparison and branch instructions, and decodes the instruction held by the instruction memory 1002 , and controls the SIMD calculating means 1004 , the VLC processing means 1005 , the external data interface 1006 , the video input device 1009 and the video output device 1010 .
- the video input device 1009 inputs the video signals from the outside, and the video output device 1010 outputs the video data to the outside.
- the image data inputted by the video input device 1009 is transferred to the external memory 1011 , and at the next step, is transferred to the external data interface 1006 according to the processing performed by the SIMD calculating means 1004 .
- the SIMD calculating means 1004 performs motion compensation, DCT and quantization processing, and acquires transformed coefficient data.
- the VLC conversion means 1005 the transformed coefficient data is encoded in variable-length encoding by the VLC transforming means, and the bit stream is generated.
- the SIMD calculating means 1004 which comprises eight parallel pipeline calculators, is capable of efficiently performing routine processing such as DCT.
- the signal-processing apparatus comprising a combination of the SIMD data-parallel processor and the MIMD data-parallel processor is typified by the above-described Document 1 is flexible toward various coding algorithms.
- the signal-processing apparatus can sufficiently handle image processing by enhancing the degree of parallelism. This is because the prior motion detection processing is for macro-block sizes of not less than 8 ⁇ 8 pels and not more than 16 ⁇ 16 pels.
- the processing efficiency of the calculators does not improve even if 16 or more parallel calculators are provided.
- the filter parameter is calculated in the unit of sub-macro-blocks of 4 ⁇ 4 pels, and filtering processing is performed based on the result.
- filtering processing can be performed in parallel, the calculators cannot be effectively used in determination processing.
- motion compensation of 1 ⁇ 4 pixel precision is introduced, and it is necessary to perform 6-tap filtering processing for the pixel generation of a half pel. Further, since the sub-macro-block size of 4 ⁇ 4 pels is introduced, up to 16 motion vectors per macro-block can be set.
- the motion detection processing in which with the small sub-macro-block size, a search of 1 ⁇ 4 pixel precision is performed and up to 16 motion vectors per macro-block are calculated is drastically increased in processing amount.
- SIMD data-parallel processor For the SIMD data-parallel processor to perform such motion detection processing, it is necessary to enhance the degree of parallelism of the calculators and set the operating frequency to a high value.
- the capability of the SIMD parallel data processor having such capability is more than enough in the decoding processing; therefore, the entire processor can not be efficiently used.
- An object of the present invention is to provide a signal-processing apparatus capable of performing high-performance and high-efficiency image processing for image processing requiring a large data processing amount like the coding/decoding processing of the MPEG-4 AVC, and an electronic apparatus using the same.
- a first aspect of the present invention provides a signal-processing apparatus comprising: an instruction-parallel processor; a data-parallel processor; and a plurality of pieces of dedicated hardware, wherein the instruction-parallel processor performs audio compression/decompression and non-routine or less-heavy operation of image compression/decompression, wherein the data-parallel processor performs, of the image compression/decompression, routine or heavy operation, and wherein the plurality of pieces of dedicated hardware perform, of the image compression/decompression, comparatively heavy processing.
- the signal-processing apparatus is composed of the instruction-parallel processor, the data-parallel processor and the dedicated hardware.
- the instruction-parallel processor performs non-routine processing of the audio compression/decompression and the image processing
- the data-parallel processor performs the routine processing of the image processing
- the dedicated hardware performs processing such as motion detection, variable-length encoding, and de-blocking filtering processing. Consequently, for signal processing of an image compression and decompression algorithm with a large processing amount, the load is distributed between software and hardware, so that a signal-processing apparatus having high processing capability and flexibility can be realized.
- a second aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, further comprising: a first instruction bus; a first data bus; a first shared memory; and an input and output interface, wherein each of the instruction-parallel processor, the data-parallel processor, the plurality of pieces of dedicated hardware and the input and output interface comprises a local memory, the instruction-parallel processor, the data-parallel processor and the plurality of pieces of dedicated hardware are connected to the first instruction bus, wherein an instruction for the instruction-parallel processor to control the data-parallel processor and the plurality of hardware is communicated through the first instruction bus, and wherein the local memory of the instruction-parallel data processor, the local memory of the data-parallel processor, the local memories of the plurality of pieces of dedicated hardware, the first shared memory and the local memory of the input and output interface are connected to the first data bus, and data transfer is performed among these memories.
- the bus traffic is distributed by separating the instruction bus and the data bus; therefore, the processing performance can be improved.
- a third aspect of the present invention provides a signal-processing apparatus according to the second aspect of the present invention, further comprising: a second data bus; a second shared memory; and a bridge unit connecting the first data bus and the second data bus, wherein the local memory of the data-parallel processor, the local memories of the plurality of pieces of dedicated hardware, the first shared memory and the local memory of the input and output interface are connected to the first data bus, and data transfer is performed among these memories, wherein the local memory of the instruction-parallel processor and the second shared memory are connected to the second data bus, and data transfer is performed between these memories, and wherein data transfer between the memories connected to the first data bus and the memories connected to the second data bus is performed through the bridge unit.
- the local memory of the data processor, the dedicated memory of the dedicated hardware and the shared memory are connected by the first data bus, and the local memory of the instruction-parallel processor and the shared memory are connected by the second data bus.
- a fourth aspect of the present invention provides a signal-processing apparatus according to the third aspect of the present invention, further comprising a control processor, wherein the instruction-parallel processor controls the data-parallel processor and the plurality of pieces of dedicated hardware through the control processor.
- the instruction-parallel processor is capable of controlling the data-parallel processor and the dedicated hardware through the control processor, the load is distributed between the instruction-parallel processor and the control processor; therefore, higher processing performance can be realized.
- a fifth aspect of the present invention provides a signal-processing apparatus according to the fourth aspect of the present invention, further comprising a second instruction bus, wherein the instruction-parallel processor, the control processor and a part of the plurality of pieces of dedicated hardware are connected to the first instruction bus, wherein the control processor, the data-parallel processor and the remainder of the plurality of pieces of dedicated hardware, the remainder being not connected to the first instruction bus, are connected to the second instruction bus, and wherein the instruction-parallel processor controls the part of the plurality of pieces of dedicated hardware, and controls, through the control processor, the data-parallel processor and the remainder of the plurality of pieces of hardware.
- the instruction-parallel processor needs to control only the control processor and part of the dedicated hardware through the first instruction bus, and the data-parallel processor that performs routine processing and the dedicated hardware are controlled by the control processor through the second instruction bus, instruction confliction of the instruction bus can be avoided; therefore, signal processing can be performed with efficiency.
- a sixth aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein the data-parallel processor comprises a plurality of processing units, and wherein a number of the plurality of processing units of the data-parallel processor is determined according to a compressed or decompressed image size.
- the degree of parallelism of the data-parallel processor is changed according to the size of the image which is the object of compression and decompression, a signal-processing apparatus capable of handling various image sizes with the same processor architecture can be provided.
- a seventh aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein the data-parallel processor comprises a plurality of processing units, and wherein a number of the plurality of processing units of the data-parallel processor is determined according to at least one of a power supply voltage and an operating frequency.
- the degree of parallelism of the data-parallel processor can be changed according to the power supply voltage and the operating frequency shared with LSIs.
- the operation frequency is reduced by increasing the degree of parallelism of the data parallel processing and the power consumption of the signal processing can be decreased; therefore, applications to electronic apparatuses such as mobile terminals particularly effective.
- An eighth aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein processing performed by the plurality of pieces of dedicated hardware includes at least one of variable-length coding processing, variable-length decoding processing, video input and output processing, motion detection processing, motion compensation processing, DCT (discrete cosine transform) processing, inverse DCT processing, quantization processing, inverse quantization processing and de-blocking filtering processing.
- processing performed by the plurality of pieces of dedicated hardware includes at least one of variable-length coding processing, variable-length decoding processing, video input and output processing, motion detection processing, motion compensation processing, DCT (discrete cosine transform) processing, inverse DCT processing, quantization processing, inverse quantization processing and de-blocking filtering processing.
- DCT discrete cosine transform
- incensement of the motion frequency for the instruction-parallel processor and the data-parallel processor can be controlled by processing the heavy modules of the processing amount such as motion detection, variable-length encoding/decoding, and de-blocking filter by the dedicated hardware.
- a ninth aspect of the present invention provides a signal-processing apparatus according to the fifth aspect of the present invention, wherein processing performed by the part, of the plurality of pieces of dedicated hardware, the part being connected to the first instruction bus, is variable-length coding processing and/or variable-length decoding processing.
- variable-length encoding and/or decoding can be directly and frequently controlled by the instruction-parallel processor. Consequently, the variable-length encoding and/or decoding can be finely and diversely controlled.
- a tenth aspect of the present invention provides an electronic apparatus comprising a signal-processing apparatus, the signal-processing apparatus comprising: an instruction-parallel processor; a data-parallel processor; and a plurality of pieces of dedicated hardware, wherein the instruction-parallel processor performs audio compression/decompression and non-routine or less-heavy operation of image compression/decompression, wherein the data-parallel processor performs, of the image compression/decompression, routine or heavy operation, wherein the plurality of pieces of dedicated hardware perform, of the image compression/decompression, comparatively heavy processing, and wherein the signal-processing apparatus performs at least one of audio compression processing, audio decompression processing, image compression processing and image decompression processing.
- the electronic apparatus which makes full use of the characteristics of the signal-processing apparatus, can be provided.
- An eleventh aspect of the present invention provides an electronic apparatus according to the tenth aspect of the present invention, further comprising: a reproducer; a demodulator/error corrector; a memory; and a plurality of D/A converters, wherein the reproducer reproduces modulated coded signals from a recording medium loaded therein, wherein the demodulator/error corrector demodulates the modulated coded signals reproduced by the reproducer, error-corrects the demodulated signals, and outputs the error-corrected signals as coded data, wherein the signal-processing apparatus decodes the coded data outputted by the demodulator/error corrector, and outputs the decoded data as video data and audio data, wherein the memory stores data before decoding, during decoding and after decoding, and wherein the plurality of D/A converters D/A-convert the video data and the audio data outputted by the signal-processing apparatus, and outputs an analog video output and an analog audio output.
- coded data can be efficiently decoded at high speed, and a reproduction electronic apparatus with a low power consumption can be realized.
- a twelfth aspect of the present invention provides an electronic apparatus according to the tenth aspect of the present invention, further comprising: a plurality of A/D converters; a memory; an error corrector/modulator; and a recorder, wherein the plurality of A/D converters A/D convert an inputted analog video input and analog audio input, and outputs video data and audio data, wherein the signal-processing apparatus encodes the video data and the audio data outputted by the plurality of A/D converters, and outputs coded data, wherein the memory stores data before encoding, during encoding and after encoding, wherein the error corrector/modulator adds an error correcting code to the coded data encoded by the signal-processing apparatus, modulates the coded data, and outputs the modulated data as coded signals, and wherein the recorder records the coded signals outputted by the error corrector/modulator onto a recording medium loaded therein.
- the AV signals can be efficiently encoded at high speed, and the recording electronic apparatus with a low power consumption can be realized.
- a thirteenth aspect of the present invention provides an electronic apparatus comprising the electronic apparatus according to the eleventh aspect of the present invention and the electronic apparatus according to the twelfth aspect of the present invention.
- the AV signals can be efficiently encoded/decoded at high speed, and the electronic apparatus with a low power consumption, into which a recording function and a reproduction function are integrated, can be realized.
- FIG. 1 is a block diagram of a signal-processing apparatus in a first embodiment of the present invention
- FIG. 2 is a block diagram of a signal-processing apparatus in a second embodiment of the present invention.
- FIG. 3 is a block diagram of a video encoder in a third embodiment of the present invention.
- FIG. 4 is a block diagram of a CABAC (Context Adaptive Binary Arithmetic Coding) arithmetic coding unit
- FIG. 5 shows the layout of a coding object block and adjacent blocks
- FIG. 6 explains how a motion compensation of 1 ⁇ 4 pixel precision performs
- FIG. 7 is a block diagram of a de-blocking filter in the third embodiment of the present invention.
- FIG. 8 explains how a processing sequence of the de-blocking filter performs
- FIG. 9 shows a comparison of encoding processing amount between the third embodiment of the present invention and a different method
- FIG. 10 is a block diagram of a video decoder in a fourth embodiment of the present invention.
- FIG. 11 is a block diagram of an audio encoder in a fifth embodiment of the present invention.
- FIG. 12 is a block diagram of an audio decoder in the fifth embodiment of the present invention.
- FIG. 13 is a block diagram of an AV reproduction system in a sixth embodiment of the present invention.
- FIG. 14 is a block diagram of an AV recording system in a seventh embodiment of the present invention.
- FIG. 15 is a block diagram of an AV recording/reproduction system in an eighth embodiment of the present invention.
- FIG. 16 is a block diagram of the prior signal-processing apparatus comprising a combination of the SIMD parallel data processor and the MIMD parallel data processor;
- FIG. 17 is a block diagram of the prior image processor comprising the combination of the SIMD parallel data processor and the dedicated hardware.
- FIG. 1 is the block diagram of the signal-processing apparatus in the first embodiment of the present invention.
- the signal-processing apparatus of the present embodiment comprises: an instruction-parallel processor 100 having a local memory 110 ; a first data-parallel processor 101 having a local memory 111 ; a second data-parallel processor 102 having a local memory 112 ; a motion detection unit 103 having a local memory 113 ; a de-blocking filtering unit 104 having a local memory 114 ; a variable-length coding/decoding unit 105 having a local memory 115 ; an input and output interface 106 having a local memory 116 ; a first shared memory 121 ; a first instruction bus 130 ; and a first data bus 132 .
- the processors 100 to 102 and the units 112 to 116 are connected to the first instruction bus 130 , and the local memories 110 to 116 , the first shared memory 121 and the input and output interface 106 are connected to the first data bus 132 .
- the variable-length coding/decoding unit 105 further has a bit stream input and output 135 for external apparatuses, and the input and output interface 106 has an audio input and output 136 and a video input and output 137 for external apparatuses.
- the SIMD processor adopted as the first data-parallel processor 101 and the second data-parallel processor 102 include eight processing elements, and are capable of processing eight data streams in parallel at one instruction.
- the motion detection unit 103 the de-blocking filtering unit 104 , the variable-length coding/decoding unit 105 and the input and output interface 106 are each dedicated hardware.
- the signal is stored into the first shared memory 121 from the input and output interface 106 via the first data bus 132 .
- the motion detection unit 103 calculates the motion vector based on the image data of the previous frame stored in the first shared memory 121 and the image data of the present frame.
- the first data-parallel processor 101 calculates the prediction image data by performing motion compensation processing based on the image data of the previous frame stored in the first shared memory 121 and the motion vector calculated by the motion detection unit 103 . Moreover, difference image data of the image data of the current frame with respect to the predicted image data is calculated.
- the second data-parallel processor 102 discrete-cosine-transforms the difference image data, and quantizes the obtained DCT coefficient. Moreover, the second data-parallel processor 102 inversely quantizes the quantized DCT coefficient, inversely discrete-cosine-transforms it, calculates the difference image data, and calculates reconstructed image data from the difference image data and the predicted image data processed by the first data-parallel processor 101 .
- the second data-parallel processor 102 performs the DCT processing. As described above, it is possible to cause two different data-parallel processors to perform different processing while maintaining the operating ratios thereof, whereby the performance is improved.
- the de-blocking filtering unit 104 performs de-blocking filtering processing to the reconstructing image data, removes block noise, and stores it into the first shared memory 121 .
- variable-length coding/decoding unit 105 performs variable-length coding processing using an arithmetic code on the quantized DCT coefficient and the motion vector, and outputs the coded data as a bit stream.
- the instruction-parallel processor 100 performs the overall control of the above-described various processing through the first instruction bus 130 . Moreover, the instruction-parallel processor 100 performs a coding mode determination as to whether to perform the generation of the predicted image by intra prediction coding or by inter prediction coding.
- the data transfer between the processors and the units is performed through the first data bus 132 .
- High-efficiency image processing can be realized by performing sequential processing of the image compression/decompression by the instruction-parallel processor 100 , performing routine processing of the image compression/decompression by the first data-parallel processor 101 and the second data-parallel processor 102 , and performing heavy processing such as the motion detection processing, the de-blocking filtering processing and the variable-length coding processing by the dedicated hardware as described above.
- the demarcation in sharing the object of processing between the first data-parallel processor 101 and the second data-parallel processor 102 in the present embodiment is an example, and it may be different.
- the processing of the first data-parallel processor 101 and the second data-parallel processor 102 may be performed by one data-parallel processor.
- the motion compensation processing performed by the first data-parallel processor 101 may be performed by the motion detection unit 103 .
- FIG. 2 is the block diagram of the signal-processing apparatus in the second embodiment of the present invention.
- components similar to those of FIG. 1 are denoted by the same reference numerals, and descriptions thereof are omitted.
- the signal-processing apparatus of the present embodiment further comprises, compared to the signal-processing apparatus of the first embodiment, a control processor 107 , a second shared memory 122 , a second instruction bus 131 , a second data bus 133 , and a bridge unit 120 connecting the first data bus 132 and the second data bus 133 .
- the instruction-parallel processor 100 , the control processor 107 and the variable-length coding/decoding unit 105 are connected to the first instruction bus 130 .
- the control processor 107 , the first data-parallel processor 101 , the second data-parallel processor 102 , the motion detection unit 103 and the de-blocking filtering unit 104 are connected to the second instruction bus 131 .
- the local memories 111 to 115 , the first shared memory 121 , the input and output interface 106 and the bridge unit 120 are connected to the first data bus.
- the local memory 110 , the second shared memory 122 and the bridge unit 120 are connected to the second data bus.
- the parallel processing of data is enhanced compared to that of the first embodiment.
- the control processor 107 introduced in the present embodiment controls, in response to an instruction from the instruction-parallel processor 100 , the first data-parallel processor 101 , the second data-parallel processor 102 , the motion detection unit 103 and the de-blocking filtering unit 104 through the second instruction bus 131 . Consequently, the signal-processing apparatus of the present embodiment is capable of more rapidly performing parallel processing by the data-parallel processors and the dedicated hardware.
- the second shared memory 122 of the present embodiment stores data related to the instruction-parallel processor 100 , and data accessed at a comparatively low frequency among the data handled by the components connected to the first data bus 132 .
- the structure reduces the load on the first shared memory 121 , so that the processing efficiency of the entire signal-processing apparatus is improved.
- FIG. 3 is the block diagram showing a video encoder in the third embodiment of the present invention.
- the video encoder of the present embodiment is an encoder capable of the MPEG-4 AVC. Each component is given a name adequately expressing a function of the video encoder corresponding to the MPEG-4 AVC.
- the video encoder of the present embodiment shown in FIG. 3 comprises the signal-processing apparatus of the second embodiment. Therefore, the correspondence between the components of FIG. 3 and the components of FIG. 2 will be shown first.
- the processing of a coding controller 301 and a mode switcher 303 are performed by the instruction-parallel processor 100 of FIG. 2 .
- the processing of a motion compensator 312 and a difference detector 302 are performed by the first data-parallel processor 101 of FIG. 2 .
- the processing of a 4 ⁇ 4 DCT transformer 304 , a quantizer 305 , an inverse quantizer 306 , an inverse 4 ⁇ 4 DCT transformer 307 and a reconstructor 309 are performed by the second data-parallel processor 102 of FIG. 2 .
- a variable-length coder 308 corresponds to the variable-length coding/decoding unit 105 of FIG. 2 .
- a de-blocking filter 310 corresponds to the de-blocking filtering unit 104 of FIG. 2 .
- a frame memory 311 corresponds to the first shared memory 121 of FIG. 2
- a motion detector 313 corresponds to the motion detection unit 103 of FIG. 2 .
- a video input 314 is, in the case of intra coding, discrete-cosine-transformed (orthogonally transformed) by the 4 ⁇ 4 DCT transformer 304 to obtain the DCT coefficient. Then, the DCT coefficient is quantized by the quantizer 305 .
- a real-precision DCT is adopted for the 8 ⁇ 8 block size, and a mismatch occurs unless the DCT precision is defined.
- an integral-precision DCT is applied for the 4 ⁇ 4 block size, and consequently, a mismatch due to the DCT precision does not occur.
- the quantized DCT coefficient is entropy-coded by use of an arithmetic coder at the variable-length coder 308 . Details thereof will be described later.
- variable-length coding/decoding processing will be described.
- variable-length coding In the variable-length coding of syntax elements such as the number of macro-blocks, the motion vector difference and the conversion factor, the following two entropy coding methods are selectively used: CAVLC (Context Adaptive Variable Length Coding); and CABAC (Context Adaptive Binary Arithmetic Coding).
- CAVLC Context Adaptive Variable Length Coding
- CABAC Context Adaptive Binary Arithmetic Coding
- CABAC arithmetic coding method used in the main profile.
- a line segment with a length of “1” is divided according to the probability of occurrence of the symbol to be coded, and since the divided line segment and the symbol to be coded correspond one to one to each other, coding is performed with respect to the line segment. Since the binary number representative of the line segment is a code, the segment of the line is large, that is, the higher the probability of occurrence of the symbol to be coded is, the shorter the binary number the symbol can be expressed by and consequently, the compression rate is increased. Therefore, when coding of the object block is performed, the probability of occurrence is manipulated in accordance with the context of the peripheral block so that the compression rate is increased.
- FIG. 4 is a block diagram of a CABAC arithmetic coding unit. This is referred from FIG. 7 of Document 3.
- the CABAC arithmetic coding unit shown in FIG. 4 has a context modeler 401 , a binarizer 402 and an adaptive binary arithmetic coding processor 405 .
- the adaptive binary arithmetic coding processor 405 has an occurrence probability predictor 403 and a coder 404 .
- the context modeling is a probability model when each symbol is coded.
- a context is defined for each syntax element, and arithmetic coding is performed by switching the probability table in accordance with the context.
- FIG. 5 shows the layout of the coding object block and adjacent blocks.
- the context of the coding object block C 408 is determined in accordance with the condition of adjacent blocks A 406 and B 407 .
- the decoding processing of the variable-length-coded code is a sequential processing of analyzing occurrence probability information by a decoder and performing reconstruction based on the information. Moreover, since the manipulation of the probability of occurrence is performed by use of a table, performing these coding processing and decoding processing by use of a VLIW (Very Long Instruction Word)-compliant instruction-parallel processor (in the above-described second embodiment, corresponding to the instruction-parallel processor 100 shown in FIG. 2 ) or an SIMD data-parallel processor (similarly, corresponding to the first data-parallel processor 101 or the second data-parallel processor 102 ) does not improve the processing performance.
- VLIW Very Long Instruction Word
- variable-length coder 308 shown in FIG. 3 is processed by the variable-length coding/decoding unit 105 shown in FIG. 2 , which is dedicated hardware in the present embodiment.
- the DCT coefficient quantized by the quantizer 305 is inversely quantized by the inverse quantizer 306 and is then inversely discrete-cosine-transformed by the inverse 4 ⁇ 4 DCT transformer 307 , and the image is reconstructed by the reconstructor 309 .
- de-blocking filtering processing is performed by the de-blocking filter 310 , and the pixel value is rewritten at the 4 ⁇ 4 pixel boundary. The de-blocking filtering processing will be described later.
- FIG. 6 is an explanatory view for explaining the motion compensation of ⁇ fraction ( 1 / 4 ) ⁇ pixel precision.
- Motion compensation is to construct a predicted image closer to the image to be coded, by use of information on the motion vector when a predicted image is constructed from an image referred to. Since the code amount decreases as the prediction error decreases, the MPEG-4 AVC adopts the motion compensation of 1 ⁇ 4 pixel precision.
- the motion vector comprises two parameters representative of a translational movement in the unit of blocks (the distance moved in the horizontal direction and the distance moved in the vertical direction).
- the predicted image of the reference image pointed by the motion vector is obtained by the following manner:
- pixels A, B, C, D, E, F, G, H, I, J, K, L, M, N, P, Q, R, S, T and U are pixels in integral positions
- pixels aa, bb, cc, dd, ee, ff, gg and hh and pixels b, h, j, m and s are pixels of ⁇ fraction ( 1 / 2 ) ⁇ precision
- pixels a, c, d, e, f, g, i, k, n, p, q and r are pixels of 1 ⁇ 4 precision.
- the intermediate data bl is rounded and normalized by (Equation 2) and clipped to 0 to 255, whereby the pixel b is obtained.
- b Clip (( b 1+16)/32)
- the intermediate data 11 is rounded and normalized by (Equation 4) and clipped to 0 to 255, whereby the pixel h is obtained.
- h Clip (( h 1+16)/32) [Equation 4]
- the pixels e, g, p and r of 1 ⁇ 4 precision are each obtained by a rounded average by use of two neighboring pixels as shown in (Equation 6).
- e ( b+h+ 1)/2
- g ( B+m+ 1)/2
- p ( h+s+ 1)/2
- r ( m+s+ 1)/2 [Equation 6]
- the motion vector can be set for each sub-macro-block.
- the motion vector In the case of 4 ⁇ 4 where the sub-macro-blocks are smallest, it is necessary to interpolate pixels in 16 real positions from the pixels in the integral positions by use of a 6-tap filter.
- processing can be performed in parallel. Therefore, by using the SIMD data-parallel processor as shown in the present embodiment, filtering processing can be efficiently performed.
- the de-blocking filtering processing smoothes the distortion by performing filtering on the block boundary.
- the filtering processing performed on the 4 ⁇ 4 boundaries of the image is an adaptive filtering processing in which the filter strength is adjusted to a value most suitable for each block boundary in accordance with the value of the Boundary Strength (BS). That is, the boundary strength BS is used for determining whether to perform filtering on the boundary or not and defining the maximum value of pixel value variations when filtering is performed.
- BS Boundary Strength
- FIG. 7 is a block diagram of the de-blocking filter 310 according to the third embodiment of the present invention.
- the de-blocking filter 310 of the present embodiment comprises a BS condition determination processor 602 , a memory 603 , a controller 604 and a filtering processor 605 .
- the filtering processor 605 comprises a memory 606 and filters 607 to 609 .
- the BS condition determination processor 602 calculates the boundary strength BS, determines the result, and passes a control parameter 613 to the filtering processor 605 .
- the filtering processor 605 performs filtering processing in accordance with the control parameter 613 .
- the processing of the de-blocking filter 310 is now described with reference to FIG. 8 .
- FIG. 8 shows the processing sequence of the de-blocking filter 310 according to the third embodiment of the present invention.
- the filtering processing as shown in FIG. 8 , horizontal filtering processing for boundaries [1] to [4] is performed, and then, vertical filtering processing for boundaries [5] to [8] is performed.
- the filtering processing is adaptively switched according to the quantization parameter and the pixel value as described above, with the data processor by the SIMD data-parallel processor, the BS condition determination cannot be performed in parallel, so that the calculators disposed in parallel cannot be effectively used.
- the de-blocking filtering processing by dedicated hardware comprising the BS condition determination processor 602 and the filtering processor 605 as shown in FIG. 7 , the BS calculation processing and the filtering processing can be separately performed and this speeds up the BS condition determination processing so that the filtering processing can be performed in parallel. Consequently, the de-blocking filtering processing can be efficiently performed.
- the filter processor is capable of parallel operation, and the introduction of calculators can further reduce the number of processing cycles. This is why the de-blocking filter 310 shown in FIG. 3 is processed by the de-blocking filtering unit 104 shown in FIG. 2 which is dedicated hardware in the present embodiment.
- the image having undergone the de-blocking filtering processing by the de-blocking filter 310 in the video encoder of the present embodiment shown in FIG. 3 is stored in the frame memory 311 because it is not only used as the output image but also is referred to as the reference image for the frame and succeeding frames.
- the processing amount required when the video encoder shown in FIG. 3 is implemented by the signal-processing apparatus of the present embodiment is compared with the processing amount required when it is implemented by a different method.
- FIG. 9 is a bar chart showing the comparison of the encoding processing amount between the third embodiment of the present invention and a different method.
- a method 1 is a case where the video encoder shown in FIG. 3 is structured by use of a processor capable of issuing one instruction per clock cycle and all the processing is performed via software.
- a method 2 is a case where the video encoder shown in FIG. 3 is structured by a combination of an MIMD parallel data processor and an SIMD parallel data processor and all the processing is performed via software.
- a method 3 is a case where the video encoder shown in FIG. 3 is structured by use of an SIMD parallel data processor and dedicated VLC hardware.
- a method 4 is a case where the video encoder shown in FIG. 3 is structured by a VLIW parallel data processor, an SIMD parallel data processor and dedicated hardware, and corresponds to the present embodiment.
- the VLIW parallel data processor of the method 4 corresponds to the instruction-parallel processor 100 of the present embodiment shown in FIG. 2
- the SIMD parallel data processor of the method 4 corresponds to the first data-parallel processor 101 and the second data-parallel processor 102 of the present embodiment shown in FIG. 2
- the dedicated hardware of the method 4 corresponds to the motion detection unit 103 , the de-blocking filtering unit 104 and the variable-length coding/decoding unit 105 of the present embodiment shown in FIG. 2 .
- the processing amount in the motion detection, the motion compensation, the variable-length coding and the de-blocking filtering is large.
- the motion detection processing is “3048” megacycles
- the variable-length coding processing is “1000” megacycles
- the de-blocking filtering processing is “321” megacycles
- the motion compensation processing is “314” megacycles
- the remaining processing is “217” megacycles.
- the total processing amount is “4900” megacycles.
- the motion detection processing is “381” megacycles
- the variable-length coding processing is “333” megacycles
- the de-blocking filtering processing is “107” megacycles
- the motion compensation processing is “39” megacycles
- the remaining processing is “52” megacycles.
- the total processing amount is “900” megacycles.
- the motion detection processing is “381” megacycles
- the variable-length coding processing is “67” megacycles
- the de-blocking filtering processing is “80” megacycles
- the motion compensation processing is “39” megacycles
- the remaining processing is “30” megacycles.
- the total processing amount is “607” megacycles.
- the motion detection processing is “203” megacycles
- the variable-length coding processing is “67” megacycles
- the de-blocking filtering processing is “21” megacycles
- the motion compensation processing is “21” megacycles
- the remaining processing is “29” megacycles.
- the total processing amount is “352” megacycles.
- the motion detection processing is a processing of selecting a position (motion vector) where the sum of the absolute values of the differences between the pixel values of the object macro-block and the reference macro-block is the smallest.
- the motion vector can be set in the unit of 4 ⁇ 4 sub-macro-blocks.
- the motion detection processing is performed by an 8-parallel SIMD parallel data processor, and compared to the method 1 , a significant speedup is realized.
- the motion detection processing is performed by 16-parallel dedicated hardware capable of calculating the sum of the absolute values of the differences, higher-speed processing than the SIMD parallel data processor is realized.
- the motion compensation processing is a processing of obtaining the reference image pointed by the motion vector, with 1 ⁇ 4 pixel precision.
- parallel processing is also possible because processing is performed in the unit of 4 ⁇ 4 sub-macro-blocks.
- the motion compensation processing is performed by the 8-parallel SIMD parallel data processor in the methods 2 and 3 and by the dedicated hardware in the method 4 , thereby a significant speedup is realized.
- the variable-length coding processing which is an arithmetic coding processing called CABAC is a sequential processing of performing decoding by changing the probability of occurrence of the object block in accordance with the context of the peripheral block.
- the method 2 is intended to perform the variable-length coding processing by using an MIMD parallel data processor capable of issuing four instructions, and the processing amount is, at most, 1 ⁇ 3 that of the one instruction issuing processor of the method 1 .
- the VLC processing is performed by dedicated hardware, and since the determination processing and the table search processing are performed at high speed, the processing time can be reduced to ⁇ fraction ( 1 / 15 ) ⁇ that of the method 1 .
- the de-blocking filtering processing is a parallel processing by the MIMD parallel data processor in the method 2 , and a parallel processing by the SIMD parallel data processor in the method 3 . Since the performance of the filtering processing and the performance of the BS determination processing are not improved in the MIMD type and the SIMD type, respectively, the processing time can be reduced only to 1 ⁇ 3 to ⁇ fraction ( 1 / 4 ) ⁇ . On the other hand, in the method 4 , the de-blocking filtering processing is performed by the dedicated hardware, and by dividing the BS determination processing, and the filtering processing, and by performing of a pipeline operation, the processing time can be reduced to ⁇ fraction (1/15) ⁇ that of the method 1 .
- FIG. 10 is a block diagram of a video decoder according to a fourth embodiment of the present invention.
- the video decoder of the present embodiment is a decoder capable of the MPEG-4 AVC.
- Each component is given a name adequately expressing a function of the video decoder according to the MPEG-4 AVC.
- the video decoder of the present embodiment shown in FIG. 10 comprises the signal-processing apparatus of the second embodiment.
- the correspondence between the components of FIG. 10 and the components of FIG. 2 is now shown.
- the processing of a decoding controller 331 is performed by the instruction-parallel processor 100 of FIG. 2 .
- the processing of a motion vector decoder 336 and a motion compensator 337 are performed by the first data-parallel processor 101 of FIG. 2 .
- the processing of an inverse quantizer 333 , an inverse 4 ⁇ 4 DCT transformer 334 and a reconstructor 335 are performed by the second data-parallel processor 102 of FIG. 2 .
- a variable-length decoder 332 corresponds to the variable-length encoding/decoding unit 105 of FIG. 2
- a de-blocking filtering 338 corresponds to the de-blocking filtering unit 104 of FIG. 2
- a frame memory 339 corresponds to the first shared memory 121 of FIG. 2 .
- a encoding video input 341 encoded by arithmetic encoding is inputted to the variable-length decoder 332 and decoded to obtain the quantized DCT coefficient and the motion vector difference.
- the obtained quantized DCT coefficient is inversely quantized by the inverse quantizer 333 , and then, inversely discrete-cosine-transformed by the inverse 4 ⁇ 4 DCT transformer 334 to obtain the difference image data.
- the motion vector is obtained by the motion vector decoder 336 from the motion vector difference obtained by the variable-length decoder 332
- the predicted image is obtained by the motion compensator 337 from the reference image and the motion vector stored in the frame memory 339 .
- a new image is reconstructed by the reconstructor 335 from the difference image data and the predicted image and outputted as a video output 342 .
- the outputted video output 342 is, at the same time, de-blocking-filtering-processed by the de-blocking filter 338 , and then, stored into the frame memory 339 .
- the control of the quantizer 333 and the inverse 4 ⁇ 4 DCT transformer 334 is performed by the decoding controller 331 .
- de-blocking filtering processing the inverse quantization processing and the inverse DCT processing are similar to those in the third embodiment, and descriptions thereof are omitted.
- variable-length decoding processing and the de-blocking filtering processing by dedicated hardware, a significant speedup can be realized.
- the video decoder of the present embodiment can be realized by use of the signal-processing apparatus of the first embodiment of the present invention.
- the processing object that each processor takes charge of can be changed as required.
- FIG. 11 is a block diagram of an audio encoder according to a fifth embodiment of the present invention.
- FIG. 12 is a block diagram of an audio decoder according to the fifth embodiment of the present invention.
- an audio input 353 undergoes compression processing including sampling and quantization at a compressor 351 , undergoes encoding at a encoder 352 , and is outputted as an encoded audio output 354 .
- an encoded audio input 363 is decoded by a decoder 361 , and is inversely quantized by a decompressor 362 to be decompressed.
- Audio encoding and decoding can be processed by any processor because the required processing amount is small compared to that of video encoding and decoding according to the MPEG-4 AVC.
- the processing of the compressor 351 and the coder 352 shown in FIG. 11 and the processing of the decoder 361 and the decompressor 362 shown in FIG. 12 are performed by the instruction-parallel processor 100 shown in FIG. 1 . These processing can be performed with a sufficient margin.
- FIG. 13 is a block diagram of an AV reproduction system according to a sixth embodiment of the present invention.
- the AV reproduction system of the present embodiment has a reproducer 801 , a demodulator/error corrector 802 , an AV decoder 803 , a memory 804 , and D/A converters 805 and 807 .
- the AV decoder 803 has a video decoder 803 A and an audio decoder 803 B.
- the video decoder 803 A is the video decoder of the fourth embodiment of the present invention shown in FIG. 10 , and can be implemented by use of the signal-processing apparatus of the first embodiment of the present invention or the signal-processing apparatus of the second embodiment.
- the audio decoder 803 B is an audio decoder of the fifth embodiment of the present invention shown in FIG. 12 .
- parallel processing can be performed by the signal-processing apparatus of the first embodiment applied to the video decoder 803 A or the instruction-parallel processor 100 ( FIG. 1 or FIG. 2 ) of the signal-processing apparatus of the second embodiment, and it is unnecessary to provide a different processor. Therefore, the AV decoder 803 can be structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment.
- the reproducer 801 reproduces media on which coded AV signals are recorded, and outputs reproduction signals.
- the reproducer 801 may be any reproducer that is capable of reproducing media on which coded AV signals according to the MPEG-4 AVC standard are recorded such as a DVD video reproducer or an HD (hard disk) video reproducer.
- the demodulator/error corrector 802 demodulates the signal reproduced by the reproducer 801 , error-corrects the demodulated signal, and outputs the error-corrected signal to the AV decoder 803 .
- the video decoder 803 A of the AV decoder 803 decodes the coded video signal and outputs the decoded signal, and the outputted signal is converted to an analog signal by the D/A converter 805 and outputted as a video output 806 .
- the audio decoder 803 B of the AV decoder 803 decodes the coded audio signal and outputs the decoded signal, and the outputted signal is converted to an analog signal by the D/A converter 807 and outputted as an audio output 808 .
- AV signals before decoding, during decoding and/or after decoding, and other data are stored.
- part or all of the functions of the demodulator/error corrector 802 may be provided to the reproducer 801 .
- the AV reproduction system of the present embodiment can be used for receiving MPEG-4 AVC-compliant AV signals transmitted from CATV, the Internet or satellite communications, and can be also used for demodulating and decoding them.
- the AV reproduction system can be performed to input the received signal to the demodulator/error corrector 802 and decode the signal by the above-described process.
- the AV reproduction system of the present embodiment can be applied as a digital television by displaying the video output on a display.
- FIG. 14 is a block diagram of an AV recording system according to a seventh embodiment of the present invention.
- the AV recording system of the present embodiment has an AV encoder 825 , an error correcting code adder/modulator 827 , a recorder 828 , a memory 826 and A/D converters 822 and 824 .
- the AV encoder 825 has a video encoder 825 A and an audio encoder 825 B.
- the video encoder 825 A is the video encoder of the third embodiment of the present invention shown in FIG. 3 , and can be implemented by use of the signal-processing apparatus of the first embodiment of the present invention or the signal-processing apparatus of the second embodiment.
- the audio encoder 825 B is an audio encoder of the fifth embodiment of the present invention shown in FIG. 11 .
- parallel processing can be performed by the instruction-parallel processor 100 ( FIG. 1 or FIG. 2 ) of the signal-processing apparatus of the first embodiment or the signal-processing apparatus of the second embodiment applied to the video encoder 825 A, and it is unnecessary to provide a different processor. Therefore, the AV encoder 825 can be structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment.
- a video input 821 is A/D converted by the A/D converter 822
- an audio input 823 is A/D converted by the A/D converter 824
- these are outputted to the A/V encoder 825 .
- the video encoder 825 A of the AV encoder 825 encodes the inputted video signal according to the MPEG-4 AVC specifications, and outputs the signal as an encoded video bit stream.
- the audio encoder 825 B encodes the inputted audio signal according to the MPEG-4 AVC specifications, and outputs the signal as an encoded audio bit stream.
- the error corrector/modulator 827 adds an error correcting code to the encoded video bit stream and the encoded audio bit stream outputted by the AV encoder 825 , modulates the bit streams, and outputs them to the recorder.
- the recorder 828 records the modulated AV signal onto a recording medium.
- the recording medium includes an optical medium such as a DVD, a magnetic recording medium such as an HD (hard disk) or a semiconductor memory.
- AV signals before encoding, during encoding and/or after encoding by the AV encoder 825 , and other data are stored.
- part or all of the functions of the error corrector/modulator 827 may be included in the recorder 828 .
- the AV recording system of the present embodiment can be used as a video camera system in which a video camera is connected to an input and the signal therefrom is encoded and recorded according to the MPEG-4 AVC specifications.
- FIG. 15 is a block diagram of an AV recording/reproduction system according to an eighth embodiment of the present invention.
- the AV recording/reproduction system of the present embodiment has a controller 840 , a recorder/reproducer 841 , a modem/error processor 842 , an AV encoder/decoder 843 , an AV interface 845 and a memory 844 .
- the AV encoder/decoder 843 has a video encoder/decoder 843 A and an audio encoder/decoder 843 B.
- the AV interface 845 has video input and output, and audio input and output.
- the AV encoder/decoder 843 has functions equal to those of the video encoder of the third embodiment of the present invention, the video decoder of the fourth embodiment and the audio encoder and the audio decoder of the fifth embodiment, and is structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment. Descriptions of the operation thereof are omitted in this embodiment because they have already been given.
- the recorder/reproducer 841 records/reproduces modulated AV signals according to the MPEG-4 AVC specifications.
- the recording medium includes an optical medium such as a DVD, a magnetic recording medium such as an HD (hard disk) or a semiconductor memory.
- the recorder/reproducer 841 has a different recording/reproduction mechanism according to the recording medium being used.
- the modem/error processor 842 at the time of recording, adds an error correcting code to the video bit stream and the audio bit stream encoded by the AV encoder/decoder 843 , modulates the bit streams, and transmits them to the recorder/reproducer 841 .
- the modem/error processor 842 at the time of reproduction, demodulates the AV signal reproduced by the recorder/reproducer 841 , error-corrects the demodulated signal, and then, transmits the video bit stream and the audio bit stream to the AV encoder/decoder 843 .
- the AV interface 845 at the time of reproduction, D/A converts the video signal and the audio signal decoded by the AV encoder/decoder 843 , and outputs a video output 846 and an audio output 848 .
- the AV interface 845 at the time of recording, A/D converts a video input 847 and an audio input 849 , and transmits them to the AV encoder/decoder 843 .
- AV signals before encoding, during encoding and/or after encoding and AV signals before decoding, during decoding and/or after decoding by the AV encoder/decoder 843 , and other data are stored.
- the controller 840 controls the recorder/reproducer 841 , the modem/error processor 842 , the AV encoder/decoder 843 and the AV interface 845 to switch the functions thereof between at the time of recording and at the time of reproduction, and controls data transfer.
- part or all of the functions of the modem/error processor 842 may be included in the recorder/reproducer 841 .
- the signal-processing apparatus of the present invention and an electronic apparatus using the same are expected to be applied to various electronic apparatuses to which the MPEG-4 AVC encoding standard is applied.
- the application to electronic apparatuses is over a wide range from domestic stationary terminals to battery-driven mobile terminals such as DVD systems, video camera systems and picture-phone systems for mobile telephones currently performed according to the MPEG-2.
- the performance required of the LSI realizing the MPEG-4 AVC standard differs according to the manner of system application.
- processing performance is important, whereas for mobile terminals, reduction in power consumption is important to increase the battery life.
- the signal-processing apparatus of the present invention and an electronic apparatus using the same are applicable to both of them. That is, by combining the instruction-parallel processor, the data-parallel processor and the dedicated hardware, the improvement in processing performance and the reduction in power consumption are enabled.
- the signal-processing apparatus of the present invention comprises a plurality of SIMD processors (in the example of FIG. 1 , the first data-parallel processor 101 and the second data-parallel processor 102 ).
- One SIMD processor includes eight processing elements, and eight data streams can be processed in parallel at one instruction.
- the degree of parallelism can be made 16 , so that a low-voltage operation and the reduction in operating frequency are enabled.
- the gist of the present invention is to realize the improvement in processing performance and the reduction in power consumption by combining the instruction-parallel processor, the data-parallel processor and the dedicated hardware, and various applications are possible without departing from the gist of the invention.
- a signal-processing apparatus capable of performing high-performance and high-efficiency image processing for image processing requiring a large data processing amount like the encoding/decoding processing of the MPEG-4 AVC, and an electronic apparatus using the same can be provided.
Abstract
Description
- 1. Field of the Invention
- The present invention relates to a signal-processing apparatus that performs audio and image compression/decompression at high speed by use of a parallel processor and dedicated hardware, and an electronic apparatus using the same.
- 2. Description of the Related Art
- In response to the recent trend toward higher performance and downsizing of image processing apparatuses and image display apparatuses that handle moving images, the ISO (International Organization for Standardization) and the ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) are co-planning the standardization of MPEG-4 AVC (Advanced Video Coding) as a next-generation compression and decompression technology. The MPEG-4 AVC realizes a high image compression rate by introducing new technologies such as integer conversion of 4×4 pixels, intra prediction at up to nine directions, seven kinds of sub-macro-block types, up to 16 motion vectors per macro-block, multi-frame reference, a de-blocking filter in the loop and arithmetic coding, and aims at a code amount compressed to 50% of the MPEG-2 that has already been put into practical use.
- However, the newly introduced coding tools adopt algorithms attaching importance to the coding efficiently; therefore, the processing amount is large and mounting to the built-in system is difficult.
- For the prior signal-processing apparatus that performs compression and decompression with the encoding method, parallel processing by the processor and the structure by the dedicated hardware have been used.
- An example of the speed-enhanced signal processing using the parallel processing method by a processor is Document 1 (Japanese Patent Application No. H03-508269). The example shown in
Document 1 is a parallel processor comprising a combination of a parallel data processor of the SIMD (Single Instruction Multiple Data) type in which the number of control streams is one and the number of data streams to be processed is more than one and a parallel data processor of the MIMD (Multiple Instruction Multiple Data) type in which the number of control streams and the number of data streams are both more than one. -
FIG. 16 , which is referred fromFIG. 1 ofDocument 1, is a block diagram illustrating a signal-processing apparatus combining a prior SIMDparallel data processor 902 and an MIMDparallel data processor 903. - The signal-processing apparatus comprises a
system controller 901 that controls the entire processor, the SIMDparallel data processor 902, the MIMDparallel data processor 903, a sharedmemory bus 904 and a sharedmemory 905. - The
system controller 901 performs execution of application programs. - The SIMD
parallel data processor 902 comprises an overall controller 910,calculators 911 to 914 andlocal memories 915 to 918 respectively. One calculator and one local memory constitute one processor. The overall controller 910 executes the program, and issues the same instruction to all of thecalculators 911 to 914. Thecalculators 911 to 914 process data stored inlocal memories 915 to 918 respectively based on the same issued instruction. - The MIMD
parallel data processor 903 comprises anoverall controller 920,controllers 921 to 924,calculators 925 to 928 andlocal memories controllers 921 to 924, a different instruction is issued to each of thecalculators 925 to 928, and the data stored in each of thelocal memories 929 to 932 is processed. Theoverall controller 920 performs control for synchronization and monitoring of the entire MIMDparallel data processor 903. - In the parallel data processor as described above, when the object processing is simple and the data processing amount is large, the SIMD
parallel data processor 902 performs processing, whereas when the object processing is complicated and the data processing amount is small, the MIMDparallel data processor 903 performs processing. - On the other hand, the speed enhancing method, which improves the calculation, is used by forming the most suitable calculator for the processing that is objected with the dedicated hardware. As an example thereof, Document 2 (Japanese Patent Application No. 2000-118434) discloses a technology that realizes a speedup of the processing by performing the variable-length encoding/decoding of the image processing with the dedicated hardware.
-
FIG. 17 , which is referred fromFIG. 1 ofDocument 2, is a block diagram illustrating animage processor 1001 combining the prior SIMD parallel data processor and the dedicated hardware. - The
image processor 1001 is connected to an externalvideo input device 1009, avideo output device 1010 and anexternal memory 1011 through an externalvideo data bus 1008. Theimage processor 1001 comprises aninstruction memory 1002, aprocessor 1003, SIMD calculating means 1004, VLC (Variable-Length Coding) processing means 1005, anexternal data interface 1006, and aninternal data bus 1007. - The VLC processing means 1005 comprises the dedicated hardware.
- The
processor 1003 performs scalar, bit manipulation and the issuance of comparison and branch instructions, and decodes the instruction held by theinstruction memory 1002, and controls the SIMD calculatingmeans 1004, the VLC processing means 1005, theexternal data interface 1006, thevideo input device 1009 and thevideo output device 1010. - The
video input device 1009 inputs the video signals from the outside, and thevideo output device 1010 outputs the video data to the outside. - The image data inputted by the
video input device 1009 is transferred to theexternal memory 1011, and at the next step, is transferred to theexternal data interface 1006 according to the processing performed by theSIMD calculating means 1004. The SIMD calculating means 1004 performs motion compensation, DCT and quantization processing, and acquires transformed coefficient data. At the next step, in the VLC conversion means 1005, the transformed coefficient data is encoded in variable-length encoding by the VLC transforming means, and the bit stream is generated. - The SIMD calculating means 1004, which comprises eight parallel pipeline calculators, is capable of efficiently performing routine processing such as DCT.
- The signal-processing apparatus comprising a combination of the SIMD data-parallel processor and the MIMD data-parallel processor is typified by the above-described
Document 1 is flexible toward various coding algorithms. Thus, the signal-processing apparatus can sufficiently handle image processing by enhancing the degree of parallelism. This is because the prior motion detection processing is for macro-block sizes of not less than 8×8 pels and not more than 16×16 pels. - However, according to the MPEG-4 AVC, since the smallest sub-macro-block size is 4×4 pels, with the prior signal-processing apparatus, the processing efficiency of the calculators does not improve even if 16 or more parallel calculators are provided.
- Moreover, in the arithmetic coding/decoding processing of the MPEG-4 AVC, since the processing is performed while the probability of occurrence is changed in accordance with the contexts of peripheral macro-blocks, it is necessary to perform coding bit by bit, which means the parallel processing cannot be performed. That is, with the prior signal-processing apparatus, the processing performance in the MPEG-4 AVC cannot be improved even if the degree of parallelism of the MIMD parallel data processor is enhanced.
- In de-blocking filters of the MPEG-4 AVC, the filter parameter is calculated in the unit of sub-macro-blocks of 4×4 pels, and filtering processing is performed based on the result. When an SIMD calculator is used, although filtering processing can be performed in parallel, the calculators cannot be effectively used in determination processing.
- Moreover, with the signal-processing apparatus comprising a combination of the SIMD data-parallel processor and the dedicated hardware is typified by the above-described
Document 2, although the processing performance is improved by adopting the dedicated hardware for the arithmetic coding/decoding processing that requires high processing performance, performing motion detection with the largest processing amount by the SIMD parallel data processor causes the following problem. - In the MPEG-4 AVC, motion compensation of ¼ pixel precision is introduced, and it is necessary to perform 6-tap filtering processing for the pixel generation of a half pel. Further, since the sub-macro-block size of 4×4 pels is introduced, up to 16 motion vectors per macro-block can be set. The motion detection processing in which with the small sub-macro-block size, a search of ¼ pixel precision is performed and up to 16 motion vectors per macro-block are calculated is drastically increased in processing amount.
- For the SIMD data-parallel processor to perform such motion detection processing, it is necessary to enhance the degree of parallelism of the calculators and set the operating frequency to a high value. The capability of the SIMD parallel data processor having such capability is more than enough in the decoding processing; therefore, the entire processor can not be efficiently used.
- Furthermore, even if it is attempted to improve the processing performance by enhancing the degree of parallelism of the SIMD parallel data processor, since the block size is 4×4 pels, it is impossible for the degree of parallelism to be more than 16.
- An object of the present invention is to provide a signal-processing apparatus capable of performing high-performance and high-efficiency image processing for image processing requiring a large data processing amount like the coding/decoding processing of the MPEG-4 AVC, and an electronic apparatus using the same.
- A first aspect of the present invention provides a signal-processing apparatus comprising: an instruction-parallel processor; a data-parallel processor; and a plurality of pieces of dedicated hardware, wherein the instruction-parallel processor performs audio compression/decompression and non-routine or less-heavy operation of image compression/decompression, wherein the data-parallel processor performs, of the image compression/decompression, routine or heavy operation, and wherein the plurality of pieces of dedicated hardware perform, of the image compression/decompression, comparatively heavy processing.
- According to the present structure, the signal-processing apparatus is composed of the instruction-parallel processor, the data-parallel processor and the dedicated hardware. The instruction-parallel processor performs non-routine processing of the audio compression/decompression and the image processing, the data-parallel processor performs the routine processing of the image processing, and the dedicated hardware performs processing such as motion detection, variable-length encoding, and de-blocking filtering processing. Consequently, for signal processing of an image compression and decompression algorithm with a large processing amount, the load is distributed between software and hardware, so that a signal-processing apparatus having high processing capability and flexibility can be realized.
- A second aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, further comprising: a first instruction bus; a first data bus; a first shared memory; and an input and output interface, wherein each of the instruction-parallel processor, the data-parallel processor, the plurality of pieces of dedicated hardware and the input and output interface comprises a local memory, the instruction-parallel processor, the data-parallel processor and the plurality of pieces of dedicated hardware are connected to the first instruction bus, wherein an instruction for the instruction-parallel processor to control the data-parallel processor and the plurality of hardware is communicated through the first instruction bus, and wherein the local memory of the instruction-parallel data processor, the local memory of the data-parallel processor, the local memories of the plurality of pieces of dedicated hardware, the first shared memory and the local memory of the input and output interface are connected to the first data bus, and data transfer is performed among these memories.
- According to the present structure, in addition to the characteristics of the signal-processing apparatus in the first aspect of the present invention, the bus traffic is distributed by separating the instruction bus and the data bus; therefore, the processing performance can be improved.
- A third aspect of the present invention provides a signal-processing apparatus according to the second aspect of the present invention, further comprising: a second data bus; a second shared memory; and a bridge unit connecting the first data bus and the second data bus, wherein the local memory of the data-parallel processor, the local memories of the plurality of pieces of dedicated hardware, the first shared memory and the local memory of the input and output interface are connected to the first data bus, and data transfer is performed among these memories, wherein the local memory of the instruction-parallel processor and the second shared memory are connected to the second data bus, and data transfer is performed between these memories, and wherein data transfer between the memories connected to the first data bus and the memories connected to the second data bus is performed through the bridge unit.
- According to the present structure, the local memory of the data processor, the dedicated memory of the dedicated hardware and the shared memory are connected by the first data bus, and the local memory of the instruction-parallel processor and the shared memory are connected by the second data bus. With this, the data transfer in image processing handling a large amount of data is performed mainly through the first data bus, so that the load can be shared with the second data bus to which the instruction-parallel processor performing audio processing is connected.
- A fourth aspect of the present invention provides a signal-processing apparatus according to the third aspect of the present invention, further comprising a control processor, wherein the instruction-parallel processor controls the data-parallel processor and the plurality of pieces of dedicated hardware through the control processor.
- According to the present structure, since the instruction-parallel processor is capable of controlling the data-parallel processor and the dedicated hardware through the control processor, the load is distributed between the instruction-parallel processor and the control processor; therefore, higher processing performance can be realized.
- A fifth aspect of the present invention provides a signal-processing apparatus according to the fourth aspect of the present invention, further comprising a second instruction bus, wherein the instruction-parallel processor, the control processor and a part of the plurality of pieces of dedicated hardware are connected to the first instruction bus, wherein the control processor, the data-parallel processor and the remainder of the plurality of pieces of dedicated hardware, the remainder being not connected to the first instruction bus, are connected to the second instruction bus, and wherein the instruction-parallel processor controls the part of the plurality of pieces of dedicated hardware, and controls, through the control processor, the data-parallel processor and the remainder of the plurality of pieces of hardware.
- According to the present structure, since the instruction-parallel processor needs to control only the control processor and part of the dedicated hardware through the first instruction bus, and the data-parallel processor that performs routine processing and the dedicated hardware are controlled by the control processor through the second instruction bus, instruction confliction of the instruction bus can be avoided; therefore, signal processing can be performed with efficiency.
- A sixth aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein the data-parallel processor comprises a plurality of processing units, and wherein a number of the plurality of processing units of the data-parallel processor is determined according to a compressed or decompressed image size.
- According to the present structure, since the degree of parallelism of the data-parallel processor is changed according to the size of the image which is the object of compression and decompression, a signal-processing apparatus capable of handling various image sizes with the same processor architecture can be provided.
- A seventh aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein the data-parallel processor comprises a plurality of processing units, and wherein a number of the plurality of processing units of the data-parallel processor is determined according to at least one of a power supply voltage and an operating frequency.
- According to the present structure, the degree of parallelism of the data-parallel processor can be changed according to the power supply voltage and the operating frequency shared with LSIs. The operation frequency is reduced by increasing the degree of parallelism of the data parallel processing and the power consumption of the signal processing can be decreased; therefore, applications to electronic apparatuses such as mobile terminals particularly effective.
- An eighth aspect of the present invention provides a signal-processing apparatus according to the first aspect of the present invention, wherein processing performed by the plurality of pieces of dedicated hardware includes at least one of variable-length coding processing, variable-length decoding processing, video input and output processing, motion detection processing, motion compensation processing, DCT (discrete cosine transform) processing, inverse DCT processing, quantization processing, inverse quantization processing and de-blocking filtering processing.
- According to the present structure, in the compressing/decompressing processing, incensement of the motion frequency for the instruction-parallel processor and the data-parallel processor can be controlled by processing the heavy modules of the processing amount such as motion detection, variable-length encoding/decoding, and de-blocking filter by the dedicated hardware.
- A ninth aspect of the present invention provides a signal-processing apparatus according to the fifth aspect of the present invention, wherein processing performed by the part, of the plurality of pieces of dedicated hardware, the part being connected to the first instruction bus, is variable-length coding processing and/or variable-length decoding processing.
- According to the present structure, the dedicated hardware performing variable-length encoding and/or decoding can be directly and frequently controlled by the instruction-parallel processor. Consequently, the variable-length encoding and/or decoding can be finely and diversely controlled.
- A tenth aspect of the present invention provides an electronic apparatus comprising a signal-processing apparatus, the signal-processing apparatus comprising: an instruction-parallel processor; a data-parallel processor; and a plurality of pieces of dedicated hardware, wherein the instruction-parallel processor performs audio compression/decompression and non-routine or less-heavy operation of image compression/decompression, wherein the data-parallel processor performs, of the image compression/decompression, routine or heavy operation, wherein the plurality of pieces of dedicated hardware perform, of the image compression/decompression, comparatively heavy processing, and wherein the signal-processing apparatus performs at least one of audio compression processing, audio decompression processing, image compression processing and image decompression processing.
- According to the present structure, the electronic apparatus, which makes full use of the characteristics of the signal-processing apparatus, can be provided.
- An eleventh aspect of the present invention provides an electronic apparatus according to the tenth aspect of the present invention, further comprising: a reproducer; a demodulator/error corrector; a memory; and a plurality of D/A converters, wherein the reproducer reproduces modulated coded signals from a recording medium loaded therein, wherein the demodulator/error corrector demodulates the modulated coded signals reproduced by the reproducer, error-corrects the demodulated signals, and outputs the error-corrected signals as coded data, wherein the signal-processing apparatus decodes the coded data outputted by the demodulator/error corrector, and outputs the decoded data as video data and audio data, wherein the memory stores data before decoding, during decoding and after decoding, and wherein the plurality of D/A converters D/A-convert the video data and the audio data outputted by the signal-processing apparatus, and outputs an analog video output and an analog audio output.
- According to the present structure, coded data can be efficiently decoded at high speed, and a reproduction electronic apparatus with a low power consumption can be realized.
- A twelfth aspect of the present invention provides an electronic apparatus according to the tenth aspect of the present invention, further comprising: a plurality of A/D converters; a memory; an error corrector/modulator; and a recorder, wherein the plurality of A/D converters A/D convert an inputted analog video input and analog audio input, and outputs video data and audio data, wherein the signal-processing apparatus encodes the video data and the audio data outputted by the plurality of A/D converters, and outputs coded data, wherein the memory stores data before encoding, during encoding and after encoding, wherein the error corrector/modulator adds an error correcting code to the coded data encoded by the signal-processing apparatus, modulates the coded data, and outputs the modulated data as coded signals, and wherein the recorder records the coded signals outputted by the error corrector/modulator onto a recording medium loaded therein.
- According to the present structure, the AV signals can be efficiently encoded at high speed, and the recording electronic apparatus with a low power consumption can be realized.
- A thirteenth aspect of the present invention provides an electronic apparatus comprising the electronic apparatus according to the eleventh aspect of the present invention and the electronic apparatus according to the twelfth aspect of the present invention.
- According to the present structure, the AV signals can be efficiently encoded/decoded at high speed, and the electronic apparatus with a low power consumption, into which a recording function and a reproduction function are integrated, can be realized.
- The above, and other objects, features and advantages of the present invention will become apparent from the following description read in conjunction with the accompanying drawings, in which like reference numerals designate the same elements.
-
FIG. 1 is a block diagram of a signal-processing apparatus in a first embodiment of the present invention; -
FIG. 2 is a block diagram of a signal-processing apparatus in a second embodiment of the present invention; -
FIG. 3 is a block diagram of a video encoder in a third embodiment of the present invention; -
FIG. 4 is a block diagram of a CABAC (Context Adaptive Binary Arithmetic Coding) arithmetic coding unit; -
FIG. 5 shows the layout of a coding object block and adjacent blocks; -
FIG. 6 explains how a motion compensation of ¼ pixel precision performs; -
FIG. 7 is a block diagram of a de-blocking filter in the third embodiment of the present invention; -
FIG. 8 explains how a processing sequence of the de-blocking filter performs; -
FIG. 9 shows a comparison of encoding processing amount between the third embodiment of the present invention and a different method; -
FIG. 10 is a block diagram of a video decoder in a fourth embodiment of the present invention; -
FIG. 11 is a block diagram of an audio encoder in a fifth embodiment of the present invention; -
FIG. 12 is a block diagram of an audio decoder in the fifth embodiment of the present invention; -
FIG. 13 is a block diagram of an AV reproduction system in a sixth embodiment of the present invention; -
FIG. 14 is a block diagram of an AV recording system in a seventh embodiment of the present invention; -
FIG. 15 is a block diagram of an AV recording/reproduction system in an eighth embodiment of the present invention; -
FIG. 16 is a block diagram of the prior signal-processing apparatus comprising a combination of the SIMD parallel data processor and the MIMD parallel data processor; and -
FIG. 17 is a block diagram of the prior image processor comprising the combination of the SIMD parallel data processor and the dedicated hardware. - Hereinafter, embodiments of the present invention are described with reference to the accompanying drawings.
- (First Embodiment)
-
FIG. 1 is the block diagram of the signal-processing apparatus in the first embodiment of the present invention. The signal-processing apparatus of the present embodiment comprises: an instruction-parallel processor 100 having alocal memory 110; a first data-parallel processor 101 having alocal memory 111; a second data-parallel processor 102 having alocal memory 112; amotion detection unit 103 having alocal memory 113; ade-blocking filtering unit 104 having alocal memory 114; a variable-length coding/decoding unit 105 having alocal memory 115; an input andoutput interface 106 having alocal memory 116; a first sharedmemory 121; afirst instruction bus 130; and afirst data bus 132. Theprocessors 100 to 102 and theunits 112 to 116 are connected to thefirst instruction bus 130, and thelocal memories 110 to 116, the first sharedmemory 121 and the input andoutput interface 106 are connected to thefirst data bus 132. The variable-length coding/decoding unit 105 further has a bit stream input andoutput 135 for external apparatuses, and the input andoutput interface 106 has an audio input andoutput 136 and a video input andoutput 137 for external apparatuses. - The SIMD processor adopted as the first data-
parallel processor 101 and the second data-parallel processor 102 include eight processing elements, and are capable of processing eight data streams in parallel at one instruction. - The
motion detection unit 103, thede-blocking filtering unit 104, the variable-length coding/decoding unit 105 and the input andoutput interface 106 are each dedicated hardware. - Next, the operation of the present embodiment will be described in outline with image coding processing as an example.
- After an externally inputted video signal is A/D converted, the signal is stored into the first shared
memory 121 from the input andoutput interface 106 via thefirst data bus 132. - The
motion detection unit 103 calculates the motion vector based on the image data of the previous frame stored in the first sharedmemory 121 and the image data of the present frame. - The first data-
parallel processor 101 calculates the prediction image data by performing motion compensation processing based on the image data of the previous frame stored in the first sharedmemory 121 and the motion vector calculated by themotion detection unit 103. Moreover, difference image data of the image data of the current frame with respect to the predicted image data is calculated. - The second data-
parallel processor 102 discrete-cosine-transforms the difference image data, and quantizes the obtained DCT coefficient. Moreover, the second data-parallel processor 102 inversely quantizes the quantized DCT coefficient, inversely discrete-cosine-transforms it, calculates the difference image data, and calculates reconstructed image data from the difference image data and the predicted image data processed by the first data-parallel processor 101. - In the signal-processing apparatus of the present embodiment, while the first data-
parallel processor 101 is performing the calculation of the pixel value of the motion compensation processing, the second data-parallel processor 102 performs the DCT processing. As described above, it is possible to cause two different data-parallel processors to perform different processing while maintaining the operating ratios thereof, whereby the performance is improved. - The
de-blocking filtering unit 104 performs de-blocking filtering processing to the reconstructing image data, removes block noise, and stores it into the first sharedmemory 121. - The variable-length coding/
decoding unit 105 performs variable-length coding processing using an arithmetic code on the quantized DCT coefficient and the motion vector, and outputs the coded data as a bit stream. - The instruction-
parallel processor 100 performs the overall control of the above-described various processing through thefirst instruction bus 130. Moreover, the instruction-parallel processor 100 performs a coding mode determination as to whether to perform the generation of the predicted image by intra prediction coding or by inter prediction coding. - The data transfer between the processors and the units is performed through the
first data bus 132. - High-efficiency image processing can be realized by performing sequential processing of the image compression/decompression by the instruction-
parallel processor 100, performing routine processing of the image compression/decompression by the first data-parallel processor 101 and the second data-parallel processor 102, and performing heavy processing such as the motion detection processing, the de-blocking filtering processing and the variable-length coding processing by the dedicated hardware as described above. - The demarcation in sharing the object of processing between the first data-
parallel processor 101 and the second data-parallel processor 102 in the present embodiment is an example, and it may be different. In other word, according to the performance of the processor, the processing of the first data-parallel processor 101 and the second data-parallel processor 102 may be performed by one data-parallel processor. - Further, the motion compensation processing performed by the first data-
parallel processor 101 may be performed by themotion detection unit 103. - (Second Embodiment)
-
FIG. 2 is the block diagram of the signal-processing apparatus in the second embodiment of the present invention. InFIG. 2 , components similar to those ofFIG. 1 are denoted by the same reference numerals, and descriptions thereof are omitted. - The signal-processing apparatus of the present embodiment further comprises, compared to the signal-processing apparatus of the first embodiment, a
control processor 107, a second sharedmemory 122, asecond instruction bus 131, asecond data bus 133, and abridge unit 120 connecting thefirst data bus 132 and thesecond data bus 133. - The instruction-
parallel processor 100, thecontrol processor 107 and the variable-length coding/decoding unit 105 are connected to thefirst instruction bus 130. - The
control processor 107, the first data-parallel processor 101, the second data-parallel processor 102, themotion detection unit 103 and thede-blocking filtering unit 104 are connected to thesecond instruction bus 131. - The
local memories 111 to 115, the first sharedmemory 121, the input andoutput interface 106 and thebridge unit 120 are connected to the first data bus. Thelocal memory 110, the second sharedmemory 122 and thebridge unit 120 are connected to the second data bus. - In the signal-processing apparatus of the present embodiment, the parallel processing of data is enhanced compared to that of the first embodiment. In other word, the
control processor 107 introduced in the present embodiment controls, in response to an instruction from the instruction-parallel processor 100, the first data-parallel processor 101, the second data-parallel processor 102, themotion detection unit 103 and thede-blocking filtering unit 104 through thesecond instruction bus 131. Consequently, the signal-processing apparatus of the present embodiment is capable of more rapidly performing parallel processing by the data-parallel processors and the dedicated hardware. - Further, the second shared
memory 122 of the present embodiment stores data related to the instruction-parallel processor 100, and data accessed at a comparatively low frequency among the data handled by the components connected to thefirst data bus 132. The structure reduces the load on the first sharedmemory 121, so that the processing efficiency of the entire signal-processing apparatus is improved. - The operation of the present embodiment will be described in detail in the third embodiment described below.
- (Third Embodiment)
-
FIG. 3 is the block diagram showing a video encoder in the third embodiment of the present invention. - The video encoder of the present embodiment is an encoder capable of the MPEG-4 AVC. Each component is given a name adequately expressing a function of the video encoder corresponding to the MPEG-4 AVC.
- The video encoder of the present embodiment shown in
FIG. 3 comprises the signal-processing apparatus of the second embodiment. Therefore, the correspondence between the components ofFIG. 3 and the components ofFIG. 2 will be shown first. - The processing of a
coding controller 301 and amode switcher 303 are performed by the instruction-parallel processor 100 ofFIG. 2 . - The processing of a
motion compensator 312 and adifference detector 302 are performed by the first data-parallel processor 101 ofFIG. 2 . - The processing of a 4×4
DCT transformer 304, aquantizer 305, aninverse quantizer 306, an inverse 4×4DCT transformer 307 and areconstructor 309 are performed by the second data-parallel processor 102 ofFIG. 2 . - A variable-
length coder 308 corresponds to the variable-length coding/decoding unit 105 ofFIG. 2 . Ade-blocking filter 310 corresponds to thede-blocking filtering unit 104 ofFIG. 2 . Aframe memory 311 corresponds to the first sharedmemory 121 ofFIG. 2 , and amotion detector 313 corresponds to themotion detection unit 103 ofFIG. 2 . - Next, primary signal processing of the MPEG-4 AVC will be described with reference to the operation of the components of the present embodiment.
- First, encoding processing will be described with reference to
FIG. 3 . Avideo input 314 is, in the case of intra coding, discrete-cosine-transformed (orthogonally transformed) by the 4×4DCT transformer 304 to obtain the DCT coefficient. Then, the DCT coefficient is quantized by thequantizer 305. - According to existing coding standards such as the MPEG-2 and the H.263, a real-precision DCT is adopted for the 8×8 block size, and a mismatch occurs unless the DCT precision is defined. However, according to the MPEG-4 AVC, an integral-precision DCT is applied for the 4×4 block size, and consequently, a mismatch due to the DCT precision does not occur.
- The quantized DCT coefficient is entropy-coded by use of an arithmetic coder at the variable-
length coder 308. Details thereof will be described later. - Next, variable-length coding/decoding processing will be described.
- The outline of the MPEG-4 AVC is described in
Document 3 “The overview of MPEG-4 AVCIH.264 and its standardization” (Teruaki SUZUKI, Information Processing Society of Japan, Audio Visual and Multimedia Information Processing 38-13, pp.69-73, November, 2002). Description will be given based onDocument 3. - In the variable-length coding of syntax elements such as the number of macro-blocks, the motion vector difference and the conversion factor, the following two entropy coding methods are selectively used: CAVLC (Context Adaptive Variable Length Coding); and CABAC (Context Adaptive Binary Arithmetic Coding).
- In this description, an arithmetic coding method called CABAC used in the main profile will be explained. In the arithmetic coding, a line segment with a length of “1” is divided according to the probability of occurrence of the symbol to be coded, and since the divided line segment and the symbol to be coded correspond one to one to each other, coding is performed with respect to the line segment. Since the binary number representative of the line segment is a code, the segment of the line is large, that is, the higher the probability of occurrence of the symbol to be coded is, the shorter the binary number the symbol can be expressed by and consequently, the compression rate is increased. Therefore, when coding of the object block is performed, the probability of occurrence is manipulated in accordance with the context of the peripheral block so that the compression rate is increased.
-
FIG. 4 is a block diagram of a CABAC arithmetic coding unit. This is referred fromFIG. 7 ofDocument 3. The CABAC arithmetic coding unit shown inFIG. 4 has acontext modeler 401, abinarizer 402 and an adaptive binaryarithmetic coding processor 405. The adaptive binaryarithmetic coding processor 405 has anoccurrence probability predictor 403 and acoder 404. - The context modeling is a probability model when each symbol is coded. A context is defined for each syntax element, and arithmetic coding is performed by switching the probability table in accordance with the context.
-
FIG. 5 shows the layout of the coding object block and adjacent blocks. InFIG. 5 , when a coding object block C408 is coded, the context of the coding object block C408 is determined in accordance with the condition of adjacent blocks A406 and B407. - In the above-described arithmetic coding processing, the decoding processing of the variable-length-coded code is a sequential processing of analyzing occurrence probability information by a decoder and performing reconstruction based on the information. Moreover, since the manipulation of the probability of occurrence is performed by use of a table, performing these coding processing and decoding processing by use of a VLIW (Very Long Instruction Word)-compliant instruction-parallel processor (in the above-described second embodiment, corresponding to the instruction-
parallel processor 100 shown inFIG. 2 ) or an SIMD data-parallel processor (similarly, corresponding to the first data-parallel processor 101 or the second data-parallel processor 102) does not improve the processing performance. Rather, by performing these processing by use of dedicated hardware (similarly, corresponding to the variable-length coding/decoding unit 105), the load can be distributed between the instruction-parallel processor and the data-parallel processor. Consequently, the operating frequency is reduced, so that the frequency balance of the processor can be made excellent. This is why the variable-length coder 308 shown inFIG. 3 is processed by the variable-length coding/decoding unit 105 shown inFIG. 2 , which is dedicated hardware in the present embodiment. - In
FIG. 3 , the DCT coefficient quantized by thequantizer 305 is inversely quantized by theinverse quantizer 306 and is then inversely discrete-cosine-transformed by the inverse 4×4DCT transformer 307, and the image is reconstructed by thereconstructor 309. On the reconstructed image, de-blocking filtering processing is performed by thede-blocking filter 310, and the pixel value is rewritten at the 4×4 pixel boundary. The de-blocking filtering processing will be described later. - Next, the motion compensation processing of ¼ pixel precision performed by the
motion compensator 312 ofFIG. 3 will be described with reference toFIG. 6 .FIG. 6 is an explanatory view for explaining the motion compensation of {fraction (1/4)} pixel precision. - Motion compensation is to construct a predicted image closer to the image to be coded, by use of information on the motion vector when a predicted image is constructed from an image referred to. Since the code amount decreases as the prediction error decreases, the MPEG-4 AVC adopts the motion compensation of ¼ pixel precision. The motion vector comprises two parameters representative of a translational movement in the unit of blocks (the distance moved in the horizontal direction and the distance moved in the vertical direction).
- The predicted image of the reference image pointed by the motion vector is obtained by the following manner:
- In
FIG. 6 , pixels A, B, C, D, E, F, G, H, I, J, K, L, M, N, P, Q, R, S, T and U are pixels in integral positions, pixels aa, bb, cc, dd, ee, ff, gg and hh and pixels b, h, j, m and s are pixels of {fraction (1/2)} precision, and pixels a, c, d, e, f, g, i, k, n, p, q and r are pixels of ¼ precision. - The procedure of obtaining the values of these pixels is now described. First, the pixel b of {fraction (1/2)} precision is obtained in the following manner: With the pixels E, F, G, H, I and J in the vicinity of the pixel b in the horizontal direction as variables, intermediate data b1 is generated by use of a 6-tap filter defined by (Equation 1).
b1=(E−5*F+20*G+20*H−5*I+J) [Equation 1] - Then, the intermediate data bl is rounded and normalized by (Equation 2) and clipped to 0 to 255, whereby the pixel b is obtained.
b=Clip((b1+16)/32) [Equation 2]
Here, Clip(X) is a function that clips the variable X inside the parentheses to a range of 0 to 255. That is, when the variable X is less than 0, b=0, when the variable X is in the range of 0 to 255, b=X, and when the variable X is not less than 256, b=255. - Likewise, the pixel h of ½ precision is obtained in the following manner: With the pixels A, C, G, M, R and T in the vicinity of the pixel h in the vertical direction as variables, intermediate data h1 is generated by use of a 6-tap filter defined by (Equation 3).
h1=(A−5*C+20*G+20*M−5*R+T) [Equation 3] - The intermediate data 11 is rounded and normalized by (Equation 4) and clipped to 0 to 255, whereby the pixel h is obtained.
h=Clip((h1+16)/32) [Equation 4] - The pixels a, c, d, f, i, k, n and q of ¼ precision are each obtained by a rounded average by use of two neighboring pixels as shown in (Equation 5).
a=(G+b+i)/2
c=(H+b+1)/2
d=(G+h+i)/2
f=(b+j+1)/2
i=(h+j+1)/2
k=(j+m+1)/2
n=(M+h+1)/2
q=+s+1)/2 [Equation 5] - Likewise, the pixels e, g, p and r of ¼ precision are each obtained by a rounded average by use of two neighboring pixels as shown in (Equation 6).
e=(b+h+1)/2
g=(B+m+1)/2
p=(h+s+1)/2
r=(m+s+1)/2 [Equation 6] - In the predicted image generation as described above, the motion vector can be set for each sub-macro-block. In the case of 4×4 where the sub-macro-blocks are smallest, it is necessary to interpolate pixels in 16 real positions from the pixels in the integral positions by use of a 6-tap filter. In the pixel interpolation, since there is no data dependence among pixels, processing can be performed in parallel. Therefore, by using the SIMD data-parallel processor as shown in the present embodiment, filtering processing can be efficiently performed.
- Next, the de-blocking filtering will be described.
- According to the MPEG-4 AVC, since the DCT processing is performed in the unit of 4×4 pixels, block distortion occurs at the pixel boundary. The de-blocking filtering processing smoothes the distortion by performing filtering on the block boundary. The filtering processing performed on the 4×4 boundaries of the image is an adaptive filtering processing in which the filter strength is adjusted to a value most suitable for each block boundary in accordance with the value of the Boundary Strength (BS). That is, the boundary strength BS is used for determining whether to perform filtering on the boundary or not and defining the maximum value of pixel value variations when filtering is performed.
-
FIG. 7 is a block diagram of thede-blocking filter 310 according to the third embodiment of the present invention. Thede-blocking filter 310 of the present embodiment comprises a BScondition determination processor 602, amemory 603, acontroller 604 and afiltering processor 605. Thefiltering processor 605 comprises amemory 606 andfilters 607 to 609. - In the
de-blocking filter 310 shown inFIG. 7 , the BScondition determination processor 602 calculates the boundary strength BS, determines the result, and passes acontrol parameter 613 to thefiltering processor 605. Thefiltering processor 605 performs filtering processing in accordance with thecontrol parameter 613. - The processing of the
de-blocking filter 310 is now described with reference toFIG. 8 . -
FIG. 8 shows the processing sequence of thede-blocking filter 310 according to the third embodiment of the present invention. As the filtering processing, as shown inFIG. 8 , horizontal filtering processing for boundaries [1] to [4] is performed, and then, vertical filtering processing for boundaries [5] to [8] is performed. - Filtering processing when the boundary strength BS=4 will be described. In the first filtering processing on the boundary [1] of a 4×4 sub-macro-block, with eight pixels p3, p2, p1, p0, q0, q1, q2 and q3 sandwiching the boundary [1] as the inputs, six pixels p2, p1, p0, q0, q1 and q2 are rewritten to pixels P2, P1, P0, Q0, Q1 and Q2.
- The pixels P2, P1 and P0 are switched of the filtering equation by the condition of (Equation 7), and are calculated by (Equation 8) and (Equation 9).
ap<βand |p0−q0|<4α+2
ap=|p2−p0| [Equation 7]
α:coefficient 1 calculated from quantization parameter
β: coefficient 2 calculated from quantization parameter - When the condition of (Equation 7) is satisfied, the pixels P0, P1 and P2 are obtained by (Equation 8).
P0=(p2+2*p1+2*p0+2*q0+q1+4)/8
P1=(p2+p1+p0+q0+2)/4
P2=(2*p3+3*p2+p1+p0+q0+4)/8 [Equation 8] - When the condition of (Equation 7) is not satisfied, the pixels P0, P1 and P2 are obtained by (Equation 9).
P0=(2*p1+p0+q1+2)/4
P1=p1
P2=p2 [Equation 9] - The pixels Q0, Q1 and Q2 are switched of the filtering equation by the condition of (Equation 10), and are calculated by (Equation 11) and (Equation 12).
ap<β and |p0−q0|<4α+2
aq=|q2−q0| [Equation 10]
α:coefficient 1 calculated from quantization parameter
β: coefficient 2 calculated from quantization parameter - When the condition of (Equation 10) is satisfied, the pixels Q0, Q1 and Q2 are calculated by (Equation 11).
Q0=(p1+2*p0+2*q0+2*q1+q2+4)/8
Q1=(p0+q0+q1+q2+2)/4
Q2=(2*q3+3*q2+q1+q0+p0+4)/8 [Equation 11] - When the condition of (Equation 10) is not satisfied, the pixels Q0, Q1 and Q2 are calculated by (Equation 12).
Q0=(2*q1+q0+p1+2)/4
Q1=q1
Q2=q2 [Equation 12] - When the filtering processing is adaptively switched according to the quantization parameter and the pixel value as described above, with the data processor by the SIMD data-parallel processor, the BS condition determination cannot be performed in parallel, so that the calculators disposed in parallel cannot be effectively used. Instead, by performing the de-blocking filtering processing by dedicated hardware comprising the BS
condition determination processor 602 and thefiltering processor 605 as shown inFIG. 7 , the BS calculation processing and the filtering processing can be separately performed and this speeds up the BS condition determination processing so that the filtering processing can be performed in parallel. Consequently, the de-blocking filtering processing can be efficiently performed. Further, since the brightness Y and the color difference UV are not dependent on data, the filter processor is capable of parallel operation, and the introduction of calculators can further reduce the number of processing cycles. This is why thede-blocking filter 310 shown inFIG. 3 is processed by thede-blocking filtering unit 104 shown inFIG. 2 which is dedicated hardware in the present embodiment. - The image having undergone the de-blocking filtering processing by the
de-blocking filter 310 in the video encoder of the present embodiment shown inFIG. 3 is stored in theframe memory 311 because it is not only used as the output image but also is referred to as the reference image for the frame and succeeding frames. - Next, the processing amount required when the video encoder shown in
FIG. 3 is implemented by the signal-processing apparatus of the present embodiment is compared with the processing amount required when it is implemented by a different method. -
FIG. 9 is a bar chart showing the comparison of the encoding processing amount between the third embodiment of the present invention and a different method. - In
FIG. 9 , amethod 1 is a case where the video encoder shown inFIG. 3 is structured by use of a processor capable of issuing one instruction per clock cycle and all the processing is performed via software. Amethod 2 is a case where the video encoder shown inFIG. 3 is structured by a combination of an MIMD parallel data processor and an SIMD parallel data processor and all the processing is performed via software. Amethod 3 is a case where the video encoder shown inFIG. 3 is structured by use of an SIMD parallel data processor and dedicated VLC hardware. Amethod 4 is a case where the video encoder shown inFIG. 3 is structured by a VLIW parallel data processor, an SIMD parallel data processor and dedicated hardware, and corresponds to the present embodiment. That is, the VLIW parallel data processor of themethod 4 corresponds to the instruction-parallel processor 100 of the present embodiment shown inFIG. 2 , the SIMD parallel data processor of themethod 4 corresponds to the first data-parallel processor 101 and the second data-parallel processor 102 of the present embodiment shown inFIG. 2 , and the dedicated hardware of themethod 4 corresponds to themotion detection unit 103, thede-blocking filtering unit 104 and the variable-length coding/decoding unit 105 of the present embodiment shown inFIG. 2 . - In the encoding processing, the processing amount in the motion detection, the motion compensation, the variable-length coding and the de-blocking filtering is large.
- Comparing concrete numerical values of these processing amounts among the methods are as follows: In the
method 1, the motion detection processing is “3048” megacycles, the variable-length coding processing is “1000” megacycles, the de-blocking filtering processing is “321” megacycles, the motion compensation processing is “314” megacycles, and the remaining processing is “217” megacycles. The total processing amount is “4900” megacycles. - In the
method 2, the motion detection processing is “381” megacycles, the variable-length coding processing is “333” megacycles, the de-blocking filtering processing is “107” megacycles, the motion compensation processing is “39” megacycles, and the remaining processing is “52” megacycles. The total processing amount is “900” megacycles. - In the
method 3, the motion detection processing is “381” megacycles, the variable-length coding processing is “67” megacycles, the de-blocking filtering processing is “80” megacycles, the motion compensation processing is “39” megacycles, and the remaining processing is “30” megacycles. The total processing amount is “607” megacycles. - In the
method 4, the motion detection processing is “203” megacycles, the variable-length coding processing is “67” megacycles, the de-blocking filtering processing is “21” megacycles, the motion compensation processing is “21” megacycles, and the remaining processing is “29” megacycles. The total processing amount is “352” megacycles. - The motion detection processing is a processing of selecting a position (motion vector) where the sum of the absolute values of the differences between the pixel values of the object macro-block and the reference macro-block is the smallest. In the case of the MPEG-4 AVC, the motion vector can be set in the unit of 4×4 sub-macro-blocks.
- Therefore, the calculation of the sum of the absolute values of the differences among 16 pixels can be processed in parallel. In the
methods method 1, a significant speedup is realized. In themethod 4, since the motion detection processing is performed by 16-parallel dedicated hardware capable of calculating the sum of the absolute values of the differences, higher-speed processing than the SIMD parallel data processor is realized. - The motion compensation processing is a processing of obtaining the reference image pointed by the motion vector, with ¼ pixel precision. In this processing, parallel processing is also possible because processing is performed in the unit of 4×4 sub-macro-blocks. Like in the case of the motion detection processing, the motion compensation processing is performed by the 8-parallel SIMD parallel data processor in the
methods method 4, thereby a significant speedup is realized. - The variable-length coding processing which is an arithmetic coding processing called CABAC is a sequential processing of performing decoding by changing the probability of occurrence of the object block in accordance with the context of the peripheral block. The
method 2 is intended to perform the variable-length coding processing by using an MIMD parallel data processor capable of issuing four instructions, and the processing amount is, at most, ⅓ that of the one instruction issuing processor of themethod 1. In themethods method 1. - The de-blocking filtering processing is a parallel processing by the MIMD parallel data processor in the
method 2, and a parallel processing by the SIMD parallel data processor in themethod 3. Since the performance of the filtering processing and the performance of the BS determination processing are not improved in the MIMD type and the SIMD type, respectively, the processing time can be reduced only to ⅓ to {fraction (1/4)}. On the other hand, in themethod 4, the de-blocking filtering processing is performed by the dedicated hardware, and by dividing the BS determination processing, and the filtering processing, and by performing of a pipeline operation, the processing time can be reduced to {fraction (1/15)} that of themethod 1. - As is apparent from the above, by constituting the motion detection processing, the motion compensation processing, the variable-length coding processing and the de-blocking filtering by dedicated hardware like in the present embodiment, a significant speedup is realized.
- (Fourth Embodiment)
-
FIG. 10 is a block diagram of a video decoder according to a fourth embodiment of the present invention. - The video decoder of the present embodiment is a decoder capable of the MPEG-4 AVC. Each component is given a name adequately expressing a function of the video decoder according to the MPEG-4 AVC.
- The video decoder of the present embodiment shown in
FIG. 10 comprises the signal-processing apparatus of the second embodiment. The correspondence between the components ofFIG. 10 and the components ofFIG. 2 is now shown. - The processing of a
decoding controller 331 is performed by the instruction-parallel processor 100 ofFIG. 2 . - The processing of a
motion vector decoder 336 and amotion compensator 337 are performed by the first data-parallel processor 101 ofFIG. 2 . - The processing of an
inverse quantizer 333, an inverse 4×4DCT transformer 334 and areconstructor 335 are performed by the second data-parallel processor 102 ofFIG. 2 . - A variable-
length decoder 332 corresponds to the variable-length encoding/decoding unit 105 ofFIG. 2 , ade-blocking filtering 338 corresponds to thede-blocking filtering unit 104 ofFIG. 2 , and aframe memory 339 corresponds to the first sharedmemory 121 ofFIG. 2 . - The outline of the operation of the video decoder of the present embodiment is now described.
- A
encoding video input 341 encoded by arithmetic encoding is inputted to the variable-length decoder 332 and decoded to obtain the quantized DCT coefficient and the motion vector difference. The obtained quantized DCT coefficient is inversely quantized by theinverse quantizer 333, and then, inversely discrete-cosine-transformed by the inverse 4×4DCT transformer 334 to obtain the difference image data. - On the other hand, the motion vector is obtained by the
motion vector decoder 336 from the motion vector difference obtained by the variable-length decoder 332, and the predicted image is obtained by themotion compensator 337 from the reference image and the motion vector stored in theframe memory 339. - A new image is reconstructed by the reconstructor 335 from the difference image data and the predicted image and outputted as a
video output 342. The outputtedvideo output 342 is, at the same time, de-blocking-filtering-processed by thede-blocking filter 338, and then, stored into theframe memory 339. - The control of the
quantizer 333 and the inverse 4×4DCT transformer 334 is performed by thedecoding controller 331. - The de-blocking filtering processing, the inverse quantization processing and the inverse DCT processing are similar to those in the third embodiment, and descriptions thereof are omitted.
- In the present embodiment, by performing the variable-length decoding processing and the de-blocking filtering processing by dedicated hardware, a significant speedup can be realized.
- Moreover, while the above description takes up an example in which the video decoder of the present embodiment is implemented by use of the signal-processing apparatus of the second embodiment of the present invention shown in
FIG. 2 , the video decoder of the present embodiment can be realized by use of the signal-processing apparatus of the first embodiment of the present invention. Moreover, the processing object that each processor takes charge of can be changed as required. - (Fifth Embodiment)
-
FIG. 11 is a block diagram of an audio encoder according to a fifth embodiment of the present invention.FIG. 12 is a block diagram of an audio decoder according to the fifth embodiment of the present invention. - In the audio encoder shown in
FIG. 11 , anaudio input 353 undergoes compression processing including sampling and quantization at acompressor 351, undergoes encoding at aencoder 352, and is outputted as an encodedaudio output 354. - In the audio decoder shown in
FIG. 12 , an encoded audio input 363 is decoded by adecoder 361, and is inversely quantized by adecompressor 362 to be decompressed. - Audio encoding and decoding can be processed by any processor because the required processing amount is small compared to that of video encoding and decoding according to the MPEG-4 AVC.
- When the audio encoder and the audio decoder of the present embodiment are implemented by use of the signal-processing apparatus of the first embodiment, the processing of the
compressor 351 and thecoder 352 shown inFIG. 11 and the processing of thedecoder 361 and thedecompressor 362 shown inFIG. 12 are performed by the instruction-parallel processor 100 shown inFIG. 1 . These processing can be performed with a sufficient margin. - (Sixth Embodiment)
-
FIG. 13 is a block diagram of an AV reproduction system according to a sixth embodiment of the present invention. - The AV reproduction system of the present embodiment has a
reproducer 801, a demodulator/error corrector 802, anAV decoder 803, amemory 804, and D/A converters AV decoder 803 has avideo decoder 803A and anaudio decoder 803B. - The
video decoder 803A is the video decoder of the fourth embodiment of the present invention shown inFIG. 10 , and can be implemented by use of the signal-processing apparatus of the first embodiment of the present invention or the signal-processing apparatus of the second embodiment. - The
audio decoder 803B is an audio decoder of the fifth embodiment of the present invention shown inFIG. 12 . As mentioned in the fifth embodiment, in the processing of the audio decoder of the fifth embodiment, since the required processing amount is small compared to that of the image data processing, parallel processing can be performed by the signal-processing apparatus of the first embodiment applied to thevideo decoder 803A or the instruction-parallel processor 100 (FIG. 1 orFIG. 2 ) of the signal-processing apparatus of the second embodiment, and it is unnecessary to provide a different processor. Therefore, theAV decoder 803 can be structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment. - The
reproducer 801 reproduces media on which coded AV signals are recorded, and outputs reproduction signals. Thereproducer 801 may be any reproducer that is capable of reproducing media on which coded AV signals according to the MPEG-4 AVC standard are recorded such as a DVD video reproducer or an HD (hard disk) video reproducer. - The demodulator/
error corrector 802 demodulates the signal reproduced by thereproducer 801, error-corrects the demodulated signal, and outputs the error-corrected signal to theAV decoder 803. - The
video decoder 803A of theAV decoder 803 decodes the coded video signal and outputs the decoded signal, and the outputted signal is converted to an analog signal by the D/A converter 805 and outputted as avideo output 806. - The
audio decoder 803B of theAV decoder 803 decodes the coded audio signal and outputs the decoded signal, and the outputted signal is converted to an analog signal by the D/A converter 807 and outputted as anaudio output 808. - In the
memory 804, AV signals before decoding, during decoding and/or after decoding, and other data are stored. - In the AV reproduction system of the present embodiment, part or all of the functions of the demodulator/
error corrector 802 may be provided to thereproducer 801. - The AV reproduction system of the present embodiment can be used for receiving MPEG-4 AVC-compliant AV signals transmitted from CATV, the Internet or satellite communications, and can be also used for demodulating and decoding them. In this case, the AV reproduction system can be performed to input the received signal to the demodulator/
error corrector 802 and decode the signal by the above-described process. Further, the AV reproduction system of the present embodiment can be applied as a digital television by displaying the video output on a display. - (Seventh Embodiment)
-
FIG. 14 is a block diagram of an AV recording system according to a seventh embodiment of the present invention. - The AV recording system of the present embodiment has an
AV encoder 825, an error correcting code adder/modulator 827, arecorder 828, amemory 826 and A/D converters AV encoder 825 has avideo encoder 825A and anaudio encoder 825B. - The
video encoder 825A is the video encoder of the third embodiment of the present invention shown inFIG. 3 , and can be implemented by use of the signal-processing apparatus of the first embodiment of the present invention or the signal-processing apparatus of the second embodiment. - The
audio encoder 825B is an audio encoder of the fifth embodiment of the present invention shown inFIG. 11 . As mentioned in the fifth embodiment, in the processing of the audio encoder of the fifth embodiment, since the required processing amount is small compared to that of the image data processing, parallel processing can be performed by the instruction-parallel processor 100 (FIG. 1 orFIG. 2 ) of the signal-processing apparatus of the first embodiment or the signal-processing apparatus of the second embodiment applied to thevideo encoder 825A, and it is unnecessary to provide a different processor. Therefore, theAV encoder 825 can be structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment. - The outline of the operation of the AV recording system of the present embodiment is now described.
- A video input 821 is A/D converted by the A/
D converter 822, anaudio input 823 is A/D converted by the A/D converter 824, and these are outputted to the A/V encoder 825. - The
video encoder 825A of theAV encoder 825 encodes the inputted video signal according to the MPEG-4 AVC specifications, and outputs the signal as an encoded video bit stream. Likewise, theaudio encoder 825B encodes the inputted audio signal according to the MPEG-4 AVC specifications, and outputs the signal as an encoded audio bit stream. - The error corrector/
modulator 827 adds an error correcting code to the encoded video bit stream and the encoded audio bit stream outputted by theAV encoder 825, modulates the bit streams, and outputs them to the recorder. - The
recorder 828 records the modulated AV signal onto a recording medium. The recording medium includes an optical medium such as a DVD, a magnetic recording medium such as an HD (hard disk) or a semiconductor memory. - In the
memory 826, AV signals before encoding, during encoding and/or after encoding by theAV encoder 825, and other data are stored. - In the AV recording system of the present embodiment, part or all of the functions of the error corrector/
modulator 827 may be included in therecorder 828. - The AV recording system of the present embodiment can be used as a video camera system in which a video camera is connected to an input and the signal therefrom is encoded and recorded according to the MPEG-4 AVC specifications.
- (Eighth Embodiment)
-
FIG. 15 is a block diagram of an AV recording/reproduction system according to an eighth embodiment of the present invention. The AV recording/reproduction system of the present embodiment has acontroller 840, a recorder/reproducer 841, a modem/error processor 842, an AV encoder/decoder 843, anAV interface 845 and amemory 844. The AV encoder/decoder 843 has a video encoder/decoder 843A and an audio encoder/decoder 843B. TheAV interface 845 has video input and output, and audio input and output. - As for the function, the AV encoder/
decoder 843 has functions equal to those of the video encoder of the third embodiment of the present invention, the video decoder of the fourth embodiment and the audio encoder and the audio decoder of the fifth embodiment, and is structured by one signal-processing apparatus of the first embodiment or one signal-processing apparatus of the second embodiment. Descriptions of the operation thereof are omitted in this embodiment because they have already been given. - The recorder/
reproducer 841 records/reproduces modulated AV signals according to the MPEG-4 AVC specifications. The recording medium includes an optical medium such as a DVD, a magnetic recording medium such as an HD (hard disk) or a semiconductor memory. The recorder/reproducer 841 has a different recording/reproduction mechanism according to the recording medium being used. - The modem/
error processor 842, at the time of recording, adds an error correcting code to the video bit stream and the audio bit stream encoded by the AV encoder/decoder 843, modulates the bit streams, and transmits them to the recorder/reproducer 841. The modem/error processor 842, at the time of reproduction, demodulates the AV signal reproduced by the recorder/reproducer 841, error-corrects the demodulated signal, and then, transmits the video bit stream and the audio bit stream to the AV encoder/decoder 843. - The
AV interface 845, at the time of reproduction, D/A converts the video signal and the audio signal decoded by the AV encoder/decoder 843, and outputs avideo output 846 and anaudio output 848. TheAV interface 845, at the time of recording, A/D converts avideo input 847 and anaudio input 849, and transmits them to the AV encoder/decoder 843. - In the
memory 844, AV signals before encoding, during encoding and/or after encoding and AV signals before decoding, during decoding and/or after decoding by the AV encoder/decoder 843, and other data are stored. - The
controller 840 controls the recorder/reproducer 841, the modem/error processor 842, the AV encoder/decoder 843 and theAV interface 845 to switch the functions thereof between at the time of recording and at the time of reproduction, and controls data transfer. - In the AV recording/reproduction system of the present embodiment, part or all of the functions of the modem/
error processor 842 may be included in the recorder/reproducer 841. - As described above in detail, the signal-processing apparatus of the present invention and an electronic apparatus using the same are expected to be applied to various electronic apparatuses to which the MPEG-4 AVC encoding standard is applied.
- The application to electronic apparatuses is over a wide range from domestic stationary terminals to battery-driven mobile terminals such as DVD systems, video camera systems and picture-phone systems for mobile telephones currently performed according to the MPEG-2.
- In these systems, the performance required of the LSI realizing the MPEG-4 AVC standard differs according to the manner of system application. For stationary systems, since large image sizes are handled, processing performance is important, whereas for mobile terminals, reduction in power consumption is important to increase the battery life. The signal-processing apparatus of the present invention and an electronic apparatus using the same are applicable to both of them. That is, by combining the instruction-parallel processor, the data-parallel processor and the dedicated hardware, the improvement in processing performance and the reduction in power consumption are enabled.
- The signal-processing apparatus of the present invention comprises a plurality of SIMD processors (in the example of
FIG. 1 , the first data-parallel processor 101 and the second data-parallel processor 102). One SIMD processor includes eight processing elements, and eight data streams can be processed in parallel at one instruction. By changing the number of provided SIMD processors according to the purpose for using the signal-processing apparatus, various performance requirements can be met without the LSI architecture being changed. - For example, in the signal-processing apparatus for mobile terminals requiring low power consumption, by providing two SIMD processors, the degree of parallelism can be made 16, so that a low-voltage operation and the reduction in operating frequency are enabled.
- Moreover, instead of using with the degree of parallelism being 16, it can be performed using two pairs of SIMD processors each comprising eight processing elements and cause them to perform different processing.
- By dividing the entire processor and performing parallel processing such that the second SIMD processor performs DCT processing while the first SIMD processor is performing the pixel value calculation of the motion compensation, a plurality of processing can be performed while the operating ratios are maintained. Consequently, the calculation performance can be significantly improved.
- While applications conforming to the MPEG-4 AVC standard are described in the above-described embodiments, the present invention is not limited to these applications. The gist of the present invention is to realize the improvement in processing performance and the reduction in power consumption by combining the instruction-parallel processor, the data-parallel processor and the dedicated hardware, and various applications are possible without departing from the gist of the invention.
- According to the present invention, a signal-processing apparatus capable of performing high-performance and high-efficiency image processing for image processing requiring a large data processing amount like the encoding/decoding processing of the MPEG-4 AVC, and an electronic apparatus using the same can be provided.
- Having described preferred embodiments of the invention with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims.
Claims (16)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/179,873 US10230991B2 (en) | 2003-08-21 | 2008-07-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instrcuction from the first processor |
US16/257,317 US11563985B2 (en) | 2003-08-21 | 2019-01-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instruction from the first processor |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003297227A JP4699685B2 (en) | 2003-08-21 | 2003-08-21 | Signal processing apparatus and electronic apparatus using the same |
JP2003-297227 | 2003-08-21 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/179,873 Division US10230991B2 (en) | 2003-08-21 | 2008-07-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instrcuction from the first processor |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050062746A1 true US20050062746A1 (en) | 2005-03-24 |
Family
ID=34056245
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/919,238 Abandoned US20050062746A1 (en) | 2003-08-21 | 2004-08-17 | Signal-processing apparatus and electronic apparatus using same |
US12/179,873 Active 2028-05-18 US10230991B2 (en) | 2003-08-21 | 2008-07-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instrcuction from the first processor |
US16/257,317 Active 2025-02-21 US11563985B2 (en) | 2003-08-21 | 2019-01-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instruction from the first processor |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/179,873 Active 2028-05-18 US10230991B2 (en) | 2003-08-21 | 2008-07-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instrcuction from the first processor |
US16/257,317 Active 2025-02-21 US11563985B2 (en) | 2003-08-21 | 2019-01-25 | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instruction from the first processor |
Country Status (5)
Country | Link |
---|---|
US (3) | US20050062746A1 (en) |
EP (2) | EP1509044A3 (en) |
JP (1) | JP4699685B2 (en) |
KR (2) | KR20050021233A (en) |
CN (1) | CN100401783C (en) |
Cited By (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060076529A1 (en) * | 2004-10-08 | 2006-04-13 | Coprecitec, S.L. | Electronic valve for flow regulation on a cooking burner |
US20070103347A1 (en) * | 2005-11-08 | 2007-05-10 | Samsung Electronics Co., Ltd. | Method and device for decoding syntax element in CABAC decoder |
US20070245097A1 (en) * | 2006-03-23 | 2007-10-18 | Ibm Corporation | Memory compression method and apparatus for heterogeneous processor architectures in an information handling system |
US20070285286A1 (en) * | 2006-06-08 | 2007-12-13 | Via Technologies, Inc. | Decoding of Context Adaptive Binary Arithmetic Codes in Computational Core of Programmable Graphics Processing Unit |
US20080063085A1 (en) * | 2006-09-11 | 2008-03-13 | Apple Computer, Inc. | Post-processing for decoder complexity scalability |
US20080069471A1 (en) * | 2006-09-19 | 2008-03-20 | Himax Technologies Limited | Image de-blocking method |
US20080137736A1 (en) * | 2005-01-19 | 2008-06-12 | Joseph J. Laks, Patent Operations | Method and Apparatus for Real Time Parallel Encoding |
US20090006437A1 (en) * | 2007-05-31 | 2009-01-01 | Kabushiki Kaisha Toshiba | Multi-Processor |
US20090049281A1 (en) * | 2007-07-24 | 2009-02-19 | Samsung Electronics Co., Ltd. | Multimedia decoding method and multimedia decoding apparatus based on multi-core processor |
US20090052542A1 (en) * | 2007-08-23 | 2009-02-26 | Samsung Electronics Co., Ltd. | Video decoding method and apparatus |
US20090300621A1 (en) * | 2008-05-30 | 2009-12-03 | Advanced Micro Devices, Inc. | Local and Global Data Share |
US20100232496A1 (en) * | 2004-11-09 | 2010-09-16 | Masayoshi Tojima | Decoding-processing apparatus and method |
US20100239233A1 (en) * | 2005-05-06 | 2010-09-23 | Mediatek, Inc. | Video Processing and Optical Recording |
US20100296585A1 (en) * | 2009-05-22 | 2010-11-25 | Masaru Matsuura | Decode processing apparatus, decode processing method, data distribution program, and decode processing program |
US20120093234A1 (en) * | 2007-11-13 | 2012-04-19 | Elemental Technologies, Inc. | Video encoding and decoding using parallel processors |
US20120147972A1 (en) * | 2010-12-10 | 2012-06-14 | Sony Corporation | Image decoding apparatus, image decoding method, image encoding apparatus, image encoding method, and program |
US20130114724A1 (en) * | 2011-11-07 | 2013-05-09 | Canon Kabushiki Kaisha | Image encoding method, image encoding apparatus, and related encoding medium, image decoding method, image decoding apparatus, and related decoding medium |
US20130329784A1 (en) * | 2011-05-27 | 2013-12-12 | Mediatek Inc. | Method and Apparatus for Line Buffer Reduction for Video Processing |
US8660188B2 (en) | 2006-08-08 | 2014-02-25 | Panasonic Corporation | Variable length coding apparatus, and method and integrated circuit of the same |
US8818123B2 (en) | 2009-01-20 | 2014-08-26 | Megachips Corporation | Image processing apparatus and image conversion apparatus |
US20140307793A1 (en) * | 2006-09-06 | 2014-10-16 | Alexander MacInnis | Systems and Methods for Faster Throughput for Compressed Video Data Decoding |
US9271001B2 (en) | 2011-11-07 | 2016-02-23 | Canon Kabushiki Kaisha | Image coding apparatus, image coding method, image decoding apparatus, image decoding method, and storage medium |
JP2018011258A (en) * | 2016-07-15 | 2018-01-18 | 日本電気株式会社 | Processing control device, processing control method and program |
US10237550B2 (en) * | 2010-09-30 | 2019-03-19 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10869108B1 (en) | 2008-09-29 | 2020-12-15 | Calltrol Corporation | Parallel signal processing system and method |
US11322171B1 (en) | 2007-12-17 | 2022-05-03 | Wai Wu | Parallel signal processing system and method |
Families Citing this family (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4699685B2 (en) * | 2003-08-21 | 2011-06-15 | パナソニック株式会社 | Signal processing apparatus and electronic apparatus using the same |
JP3879741B2 (en) * | 2004-02-25 | 2007-02-14 | ソニー株式会社 | Image information encoding apparatus and image information encoding method |
KR100644620B1 (en) * | 2004-07-29 | 2006-11-10 | 삼성전자주식회사 | Method and apparatus for decoding moving picture which post filtering is applied |
ES2362787T3 (en) * | 2004-08-31 | 2011-07-13 | Panasonic Corporation | DEVICE AND PROCEDURE FOR CODING MOVING IMAGES. |
JP4442891B2 (en) * | 2004-11-30 | 2010-03-31 | キヤノン株式会社 | Variable length coding apparatus and variable length coding method |
FR2888349A1 (en) * | 2005-07-06 | 2007-01-12 | St Microelectronics Sa | ADAPTATION OF BIT RATE IN A DATA PROCESSING FLOW |
KR100717055B1 (en) | 2005-11-18 | 2007-05-10 | 삼성전자주식회사 | Method of decoding bin values using pipeline architecture, and decoding apparatus therefor |
US7245242B2 (en) * | 2005-11-28 | 2007-07-17 | Conexant Systems, Inc. | Decoding systems and methods |
US7956930B2 (en) | 2006-01-06 | 2011-06-07 | Microsoft Corporation | Resampling and picture resizing operations for multi-resolution video coding and decoding |
CN101043625B (en) * | 2006-03-23 | 2010-10-20 | 国际商业机器公司 | Apparatus and method for high-speed decompression of digital data |
JP2007265130A (en) * | 2006-03-29 | 2007-10-11 | Seiko Epson Corp | Image-processing device and image-processing method |
JP4834446B2 (en) | 2006-04-11 | 2011-12-14 | 株式会社東芝 | Image processing device |
US8290044B2 (en) * | 2006-05-10 | 2012-10-16 | Qualcomm Incorporation | Instruction for producing two independent sums of absolute differences |
CN101076124B (en) * | 2006-05-18 | 2010-09-08 | 北京大学深圳研究生院 | Filter for eliminating block |
KR20080008137A (en) | 2006-07-19 | 2008-01-23 | 삼성전자주식회사 | Apparatus and method for decoding image |
JP4625903B2 (en) * | 2006-07-26 | 2011-02-02 | 国立大学法人神戸大学 | Image processor |
CN1889667B (en) * | 2006-07-26 | 2011-03-30 | 浙江大学 | Video frequency signal multi-processor parallel processing method |
US7813567B2 (en) * | 2007-04-26 | 2010-10-12 | Texas Instruments Incorporated | Method of CABAC significance MAP decoding suitable for use on VLIW data processors |
US7885473B2 (en) * | 2007-04-26 | 2011-02-08 | Texas Instruments Incorporated | Method of CABAC coefficient magnitude and sign decoding suitable for use on VLIW data processors |
KR100895298B1 (en) * | 2007-04-30 | 2009-05-07 | 한국전자통신연구원 | Apparatus, Method, Data Processing Element for efficient parallel processing of multimedia data |
KR101355375B1 (en) * | 2007-07-24 | 2014-01-22 | 삼성전자주식회사 | Method and apparatus for decoding multimedia based on multicore processor |
KR101392349B1 (en) * | 2007-08-23 | 2014-05-19 | 삼성전자주식회사 | Method and apparatus for video decoding |
JP2011034190A (en) | 2009-07-30 | 2011-02-17 | Renesas Electronics Corp | Data processing device |
CN101860650B (en) * | 2009-08-24 | 2012-12-12 | 深圳市海云天科技股份有限公司 | Image acquisition and processing method based on network scoring and system thereof |
US8254709B2 (en) * | 2009-12-16 | 2012-08-28 | Industrial Technology Research Institute | Variable length encoding device of image compression system |
JP2011166273A (en) * | 2010-02-05 | 2011-08-25 | Mitsubishi Electric Corp | Moving image encoder and moving image encoding method |
JP5589582B2 (en) * | 2010-06-14 | 2014-09-17 | コニカミノルタ株式会社 | Image processing apparatus and program |
CN101977318B (en) * | 2010-10-29 | 2012-02-08 | 西安电子科技大学 | Parallel device of DCT (Discrete Cosine Transformation) quantization and method thereof |
WO2013094199A1 (en) * | 2011-12-21 | 2013-06-27 | パナソニック株式会社 | Method for encoding image, image-encoding device, method for decoding image, image-decoding device, and image encoding/decoding device |
ES2415755B2 (en) * | 2011-12-26 | 2014-02-04 | Universidad De Alicante | FLEXIBLE PROCESSING SYSTEM OF DIGITAL IMAGES. |
US20160301945A1 (en) * | 2015-02-09 | 2016-10-13 | Hitachi Information & Telecommunication Engineering, Ltd. | Image compression/decompression device |
CN106303450A (en) * | 2016-08-29 | 2017-01-04 | 合肥康胜达智能科技有限公司 | A kind of method of video image processing |
WO2019049543A1 (en) * | 2017-09-08 | 2019-03-14 | ソニー株式会社 | Audio processing device, audio processing method, and program |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4924328A (en) * | 1987-06-30 | 1990-05-08 | Kabushiki Kaisha Toshiba | System for recording/reproducing an information source having an apparatus-specific identification signal |
US5003256A (en) * | 1989-09-07 | 1991-03-26 | Amdahl Corporation | Clock skew measurement technique |
US5355508A (en) * | 1990-05-07 | 1994-10-11 | Mitsubishi Denki Kabushiki Kaisha | Parallel data processing system combining a SIMD unit with a MIMD unit and sharing a common bus, memory, and system controller |
US5588152A (en) * | 1990-11-13 | 1996-12-24 | International Business Machines Corporation | Advanced parallel processor including advanced support hardware |
US5623312A (en) * | 1994-12-22 | 1997-04-22 | Lucent Technologies Inc. | Compressed-domain bit rate reduction system |
US20010033617A1 (en) * | 2000-04-19 | 2001-10-25 | Fumitoshi Karube | Image processing device |
Family Cites Families (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2884831B2 (en) * | 1991-07-03 | 1999-04-19 | 株式会社日立製作所 | Processing equipment |
JPH0520283A (en) * | 1991-07-11 | 1993-01-29 | Mitsubishi Electric Corp | Parallel data processor |
JPH05324018A (en) | 1992-05-20 | 1993-12-07 | Fanuc Ltd | Control circuit for pc |
DE4333947C2 (en) | 1993-10-06 | 1996-08-08 | Schroff Gmbh | Equipment cabinet |
IL110573A (en) | 1994-08-04 | 1998-10-30 | Orad Hi Tec Systems Ltd | Apparatus and method for video broadcasting |
JP3674720B2 (en) * | 1995-10-23 | 2005-07-20 | 富士通株式会社 | Data transfer method in parallel computer |
KR100280285B1 (en) * | 1996-08-19 | 2001-02-01 | 윤종용 | Multimedia processor suitable for multimedia signals |
US6058465A (en) * | 1996-08-19 | 2000-05-02 | Nguyen; Le Trong | Single-instruction-multiple-data processing in a multimedia signal processor |
KR19980044990A (en) * | 1996-12-09 | 1998-09-15 | 양승택 | Structure of Portable Multimedia Data Input / Output Processor and Its Driving Method |
JPH1165989A (en) | 1997-08-22 | 1999-03-09 | Sony Computer Entertainment:Kk | Information processor |
JP3410669B2 (en) | 1997-11-28 | 2003-05-26 | 松下電器産業株式会社 | Video and audio processing device |
EP0919953B1 (en) * | 1997-11-28 | 2004-08-18 | Matsushita Electric Industrial Co., Ltd | Audio-video decoding system |
US6145030A (en) * | 1998-03-27 | 2000-11-07 | Intel Corporation | System for managing input/output address accesses at a bridge/memory controller |
JP3474115B2 (en) | 1998-10-09 | 2003-12-08 | 株式会社クボタ | Transmission device for speed difference steering work machine |
JP2002532810A (en) * | 1998-12-15 | 2002-10-02 | インテンシス・コーポレーション | Programmable parallel computer for image processing functions and control |
WO2000043868A1 (en) | 1999-01-20 | 2000-07-27 | Hitachi, Ltd. | Data processor and device for arithmetic operation |
JP2000259579A (en) | 1999-03-11 | 2000-09-22 | Hitachi Ltd | Semiconductor integrated circuit |
JP2000311132A (en) | 1999-04-28 | 2000-11-07 | Mitsubishi Electric Corp | Method for controlling memory access between plural buses |
US6718415B1 (en) * | 1999-05-14 | 2004-04-06 | Acqis Technology, Inc. | Computer system and method including console housing multiple computer modules having independent processing units, mass storage devices, and graphics controllers |
US6473819B1 (en) * | 1999-12-17 | 2002-10-29 | International Business Machines Corporation | Scalable interruptible queue locks for shared-memory multiprocessor |
JP2002024084A (en) * | 2000-07-12 | 2002-01-25 | Mitsubishi Electric Corp | Semiconductor integrated circuit device and electronic system |
JP2002041285A (en) | 2000-07-28 | 2002-02-08 | Toshiba Corp | Data processing equipment and data processing method |
CN1139873C (en) * | 2001-01-18 | 2004-02-25 | 清华大学 | Programmable video signal processor structure based on mixed video encoding method |
US6832305B2 (en) * | 2001-03-14 | 2004-12-14 | Samsung Electronics Co., Ltd. | Method and apparatus for executing coprocessor instructions |
JP2002358288A (en) | 2001-05-31 | 2002-12-13 | Hitachi Ltd | Semiconductor integrated circuit and computer readable recording medium |
AU2002351525A1 (en) * | 2001-06-28 | 2003-03-03 | Oak Technology, Inc. | System-on-a-chip controller |
JP3564445B2 (en) * | 2001-09-20 | 2004-09-08 | 松下電器産業株式会社 | Processor, compiling device and compiling method |
JP3779580B2 (en) * | 2001-09-27 | 2006-05-31 | 株式会社東芝 | Signal processing method and apparatus, signal reproduction method and apparatus, and recording medium |
US6944746B2 (en) * | 2002-04-01 | 2005-09-13 | Broadcom Corporation | RISC processor supporting one or more uninterruptible co-processors |
US8284844B2 (en) | 2002-04-01 | 2012-10-09 | Broadcom Corporation | Video decoding system supporting multiple standards |
US7242713B2 (en) * | 2002-05-02 | 2007-07-10 | Microsoft Corporation | 2-D transforms for image and video coding |
US8090928B2 (en) * | 2002-06-28 | 2012-01-03 | Intellectual Ventures I Llc | Methods and apparatus for processing scalar and vector instructions |
EP1387259B1 (en) * | 2002-07-31 | 2017-09-20 | Texas Instruments Incorporated | Inter-processor control |
US7580584B2 (en) * | 2003-07-18 | 2009-08-25 | Microsoft Corporation | Adaptive multiple quantization |
WO2005015504A1 (en) * | 2003-08-07 | 2005-02-17 | Renesas Technology Corp. | Image processing semiconductor processor |
US20050094729A1 (en) * | 2003-08-08 | 2005-05-05 | Visionflow, Inc. | Software and hardware partitioning for multi-standard video compression and decompression |
JP4699685B2 (en) * | 2003-08-21 | 2011-06-15 | パナソニック株式会社 | Signal processing apparatus and electronic apparatus using the same |
US7167971B2 (en) * | 2004-06-30 | 2007-01-23 | International Business Machines Corporation | System and method for adaptive run-time reconfiguration for a reconfigurable instruction set co-processor architecture |
US7200723B1 (en) * | 2004-08-06 | 2007-04-03 | Xilinx, Inc. | Access to a bank of registers of a device control register interface using a single address |
US8161209B2 (en) * | 2008-03-31 | 2012-04-17 | Advanced Micro Devices, Inc. | Peer-to-peer special purpose processor architecture and method |
US11345356B2 (en) * | 2019-12-23 | 2022-05-31 | Qualcomm Incorporated | Passenger-less and driver-less vehicle operating modes |
-
2003
- 2003-08-21 JP JP2003297227A patent/JP4699685B2/en not_active Expired - Fee Related
-
2004
- 2004-08-16 EP EP20040019387 patent/EP1509044A3/en not_active Ceased
- 2004-08-16 EP EP20110178690 patent/EP2395756A2/en not_active Withdrawn
- 2004-08-17 US US10/919,238 patent/US20050062746A1/en not_active Abandoned
- 2004-08-19 KR KR1020040065256A patent/KR20050021233A/en active IP Right Grant
- 2004-08-19 CN CNB2004100641420A patent/CN100401783C/en active Active
-
2008
- 2008-07-25 US US12/179,873 patent/US10230991B2/en active Active
-
2011
- 2011-04-05 KR KR1020110031428A patent/KR101213326B1/en active IP Right Grant
-
2019
- 2019-01-25 US US16/257,317 patent/US11563985B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4924328A (en) * | 1987-06-30 | 1990-05-08 | Kabushiki Kaisha Toshiba | System for recording/reproducing an information source having an apparatus-specific identification signal |
US5003256A (en) * | 1989-09-07 | 1991-03-26 | Amdahl Corporation | Clock skew measurement technique |
US5355508A (en) * | 1990-05-07 | 1994-10-11 | Mitsubishi Denki Kabushiki Kaisha | Parallel data processing system combining a SIMD unit with a MIMD unit and sharing a common bus, memory, and system controller |
US5588152A (en) * | 1990-11-13 | 1996-12-24 | International Business Machines Corporation | Advanced parallel processor including advanced support hardware |
US5623312A (en) * | 1994-12-22 | 1997-04-22 | Lucent Technologies Inc. | Compressed-domain bit rate reduction system |
US20010033617A1 (en) * | 2000-04-19 | 2001-10-25 | Fumitoshi Karube | Image processing device |
Cited By (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060076529A1 (en) * | 2004-10-08 | 2006-04-13 | Coprecitec, S.L. | Electronic valve for flow regulation on a cooking burner |
US20140064358A1 (en) * | 2004-11-09 | 2014-03-06 | Panasonic Corporation | Decoding method for decoding an incoming bitstream and method for performing the same |
US20100232496A1 (en) * | 2004-11-09 | 2010-09-16 | Masayoshi Tojima | Decoding-processing apparatus and method |
US8406308B2 (en) * | 2004-11-09 | 2013-03-26 | Panasonic Corporation | Encoding apparatus for encoding binary signals generated from binarized multivalued syntax elements and method for performing the same |
US8867612B2 (en) * | 2004-11-09 | 2014-10-21 | Panasonic Corporation | Decoding method for decoding an incoming bitstream and method for performing the same |
US9219917B2 (en) * | 2005-01-19 | 2015-12-22 | Thomson Licensing | Method and apparatus for real time parallel encoding |
US20080137736A1 (en) * | 2005-01-19 | 2008-06-12 | Joseph J. Laks, Patent Operations | Method and Apparatus for Real Time Parallel Encoding |
US20100239233A1 (en) * | 2005-05-06 | 2010-09-23 | Mediatek, Inc. | Video Processing and Optical Recording |
US7365660B2 (en) | 2005-11-08 | 2008-04-29 | Samsung Electronics Co., Ltd. | Method and device for decoding syntax element in CABAC decoder |
US20070103347A1 (en) * | 2005-11-08 | 2007-05-10 | Samsung Electronics Co., Ltd. | Method and device for decoding syntax element in CABAC decoder |
US20070245097A1 (en) * | 2006-03-23 | 2007-10-18 | Ibm Corporation | Memory compression method and apparatus for heterogeneous processor architectures in an information handling system |
US7849241B2 (en) * | 2006-03-23 | 2010-12-07 | International Business Machines Corporation | Memory compression method and apparatus for heterogeneous processor architectures in an information handling system |
US20070285286A1 (en) * | 2006-06-08 | 2007-12-13 | Via Technologies, Inc. | Decoding of Context Adaptive Binary Arithmetic Codes in Computational Core of Programmable Graphics Processing Unit |
US7656326B2 (en) * | 2006-06-08 | 2010-02-02 | Via Technologies, Inc. | Decoding of context adaptive binary arithmetic codes in computational core of programmable graphics processing unit |
US8660188B2 (en) | 2006-08-08 | 2014-02-25 | Panasonic Corporation | Variable length coding apparatus, and method and integrated circuit of the same |
US9094686B2 (en) * | 2006-09-06 | 2015-07-28 | Broadcom Corporation | Systems and methods for faster throughput for compressed video data decoding |
US20140307793A1 (en) * | 2006-09-06 | 2014-10-16 | Alexander MacInnis | Systems and Methods for Faster Throughput for Compressed Video Data Decoding |
US20080063085A1 (en) * | 2006-09-11 | 2008-03-13 | Apple Computer, Inc. | Post-processing for decoder complexity scalability |
US8731064B2 (en) * | 2006-09-11 | 2014-05-20 | Apple Inc. | Post-processing for decoder complexity scalability |
US20080069471A1 (en) * | 2006-09-19 | 2008-03-20 | Himax Technologies Limited | Image de-blocking method |
US8190582B2 (en) | 2007-05-31 | 2012-05-29 | Kabushiki Kaisha Toshiba | Multi-processor |
US20090006437A1 (en) * | 2007-05-31 | 2009-01-01 | Kabushiki Kaisha Toshiba | Multi-Processor |
US20090049281A1 (en) * | 2007-07-24 | 2009-02-19 | Samsung Electronics Co., Ltd. | Multimedia decoding method and multimedia decoding apparatus based on multi-core processor |
US8634470B2 (en) * | 2007-07-24 | 2014-01-21 | Samsung Electronics Co., Ltd. | Multimedia decoding method and multimedia decoding apparatus based on multi-core processor |
US9131240B2 (en) | 2007-08-23 | 2015-09-08 | Samsung Electronics Co., Ltd. | Video decoding method and apparatus which uses double buffering |
US20090052542A1 (en) * | 2007-08-23 | 2009-02-26 | Samsung Electronics Co., Ltd. | Video decoding method and apparatus |
US20120093234A1 (en) * | 2007-11-13 | 2012-04-19 | Elemental Technologies, Inc. | Video encoding and decoding using parallel processors |
US10678747B2 (en) | 2007-11-13 | 2020-06-09 | Amazon Technologies, Inc. | Video encoding and decoding using parallel processors |
US9747251B2 (en) * | 2007-11-13 | 2017-08-29 | Amazon Technologies, Inc. | Video encoding and decoding using parallel processors |
US11322171B1 (en) | 2007-12-17 | 2022-05-03 | Wai Wu | Parallel signal processing system and method |
US9619428B2 (en) | 2008-05-30 | 2017-04-11 | Advanced Micro Devices, Inc. | SIMD processing unit with local data share and access to a global data share of a GPU |
US10140123B2 (en) | 2008-05-30 | 2018-11-27 | Advanced Micro Devices, Inc. | SIMD processing lanes storing input pixel operand data in local register file for thread execution of image processing operations |
US20090300621A1 (en) * | 2008-05-30 | 2009-12-03 | Advanced Micro Devices, Inc. | Local and Global Data Share |
US10869108B1 (en) | 2008-09-29 | 2020-12-15 | Calltrol Corporation | Parallel signal processing system and method |
US8818123B2 (en) | 2009-01-20 | 2014-08-26 | Megachips Corporation | Image processing apparatus and image conversion apparatus |
US20100296585A1 (en) * | 2009-05-22 | 2010-11-25 | Masaru Matsuura | Decode processing apparatus, decode processing method, data distribution program, and decode processing program |
US9179158B2 (en) * | 2009-05-22 | 2015-11-03 | Sony Corporation | Decode processing apparatus, decode processing method, data distribution program, and decode processing program |
US10244264B2 (en) * | 2010-09-30 | 2019-03-26 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10237550B2 (en) * | 2010-09-30 | 2019-03-19 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10237551B2 (en) * | 2010-09-30 | 2019-03-19 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10244233B2 (en) * | 2010-09-30 | 2019-03-26 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US10291939B2 (en) * | 2010-09-30 | 2019-05-14 | Mitsubishi Electric Corporation | Moving image encoding device, moving image decoding device, moving image coding method, and moving image decoding method |
US20120147972A1 (en) * | 2010-12-10 | 2012-06-14 | Sony Corporation | Image decoding apparatus, image decoding method, image encoding apparatus, image encoding method, and program |
US9762918B2 (en) * | 2011-05-27 | 2017-09-12 | Hfi Innovation Inc. | Method and apparatus for line buffer reduction for video processing |
US9866848B2 (en) | 2011-05-27 | 2018-01-09 | Hfi Innovation Inc. | Method and apparatus for line buffer reduction for video processing |
US20130329784A1 (en) * | 2011-05-27 | 2013-12-12 | Mediatek Inc. | Method and Apparatus for Line Buffer Reduction for Video Processing |
US9219920B2 (en) * | 2011-11-07 | 2015-12-22 | Canon Kabushiki Kaisha | Image encoding method, image encoding apparatus, and related encoding medium, image decoding method, image decoding apparatus, and related decoding medium |
US9271001B2 (en) | 2011-11-07 | 2016-02-23 | Canon Kabushiki Kaisha | Image coding apparatus, image coding method, image decoding apparatus, image decoding method, and storage medium |
US20130114724A1 (en) * | 2011-11-07 | 2013-05-09 | Canon Kabushiki Kaisha | Image encoding method, image encoding apparatus, and related encoding medium, image decoding method, image decoding apparatus, and related decoding medium |
JP2018011258A (en) * | 2016-07-15 | 2018-01-18 | 日本電気株式会社 | Processing control device, processing control method and program |
Also Published As
Publication number | Publication date |
---|---|
CN1585490A (en) | 2005-02-23 |
US10230991B2 (en) | 2019-03-12 |
US20190174146A1 (en) | 2019-06-06 |
EP1509044A2 (en) | 2005-02-23 |
JP4699685B2 (en) | 2011-06-15 |
KR101213326B1 (en) | 2012-12-18 |
JP2005070938A (en) | 2005-03-17 |
KR20110044739A (en) | 2011-04-29 |
US20080307198A1 (en) | 2008-12-11 |
US11563985B2 (en) | 2023-01-24 |
CN100401783C (en) | 2008-07-09 |
EP2395756A2 (en) | 2011-12-14 |
EP1509044A3 (en) | 2009-04-29 |
KR20050021233A (en) | 2005-03-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11563985B2 (en) | Signal-processing apparatus including a second processor that, after receiving an instruction from a first processor, independantly controls a second data processing unit without further instruction from the first processor | |
US11843794B2 (en) | CABAC decoder with decoupled arithmetic decoding and inverse binarization | |
US11758184B2 (en) | Line-based compression for digital image data | |
US8615043B2 (en) | Fixed length coding based image data compression | |
JP2004007337A (en) | Image processing apparatus and its method | |
US20050129130A1 (en) | Color space coding framework | |
US20090010326A1 (en) | Method and apparatus for parallel video decoding | |
US20060133512A1 (en) | Video decoder and associated methods of operation | |
KR101194952B1 (en) | Method and apparatus for variable accuracy inter-picture timing specification for digital video encoding | |
US20110110435A1 (en) | Multi-standard video decoding system | |
US8189672B2 (en) | Method for interpolating chrominance signal in video encoder and decoder | |
KR100636911B1 (en) | Method and apparatus of video decoding based on interleaved chroma frame buffer | |
JP5655100B2 (en) | Image / audio signal processing apparatus and electronic apparatus using the same | |
JP2010055629A (en) | Image audio signal processor and electronic device using the same | |
Shoham et al. | Introduction to video compression |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KATAOKA, TOMONORI;NISHIDA, HIDESHI;KIMURA, KOUZOU;AND OTHERS;REEL/FRAME:015951/0688 Effective date: 20040915 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: PANASONIC CORPORATION, JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD.;REEL/FRAME:022434/0348 Effective date: 20081001 |