CA2181864C

CA2181864C - Subtitle encoding/decoding method and apparatus

Info

Publication number: CA2181864C
Application number: CA 2181864
Authority: CA
Inventors: Ikuo Tsukagoshi; Noriyuki Yamashita
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 1995-08-02
Filing date: 1996-07-23
Publication date: 2005-04-19
Anticipated expiration: 2016-07-23
Also published as: AU6088296A; JPH0951502A; BR9603263B1; ATE232345T1; AU702694B2; MX9603105A; US5848217A; ES2191740T3; CN1444222A; DE69626072D1; CN1291594C; JP3326670B2; CN1144210C; EP0757484A3; BR9603263A; MY116664A; US6424792B1; EP0757484B1; KR970012599A; EP0757484A2

Abstract

Subtitle searching is achieved by encoding a subtitle to be displayed exclusively during a trick playback mode.
Addresses for the subtitle to be played back during the trick playback mode are stored on a record medium reserved for addresses. A computer-readable memory directs a computer to retrieve these addresses and locate subtitles to be displayed during the trick playback mode on the record medium. The subtitles are searched by decoding and displaying the subtitles located at the addresses on the record medium to be displayed during the trick playback mode.

Description

PATENT

SUBTITLE ENCO IN ECOIaING MET OD AND APPARATUS

2 The present invention relates to encoding and decoding 3 subtitles and, more particularly, to searching for subtitles on a 4 record medium.
Television broadcasting or video reproduction fsuch as 6 from a video disk) pravides subtitles superimposed on the video 7 image. Problematically, the subtitles are permanently combined 8 with the underlying video image and cannat be manipulated at the 9 receiving (or reproducing) end. The subtitles, for e:cample, cannot be searched for information concerning a specific scene 11 occurring in the video image or sound in its corresponding audio 12 track.
13 Compact Disc Graphics (CD-G) provide some flexibility 14 in searching subtitles because this technique records graphics in the form of subcodes. However, CD-G has a serious disadvantage 16 because this technique is limited to compact disc (CD) 17 applications, which are slow by television standards. That is, 18 the CD-G technique does not lend itself to manipulation of 19 subtitles in real-time television broadcasts or video reproductions. .
21 As will be shown with referenc:e~ to Figs. 18AT-C and 19, 22 the lead time required to generate a f~a'~1 CD-G screen is grossly 23 inadequate for normal television or video broadcasts. Fig. 18A
sony.l5\3582.app PATENT

1 depicts the CD-G data format in which one frame includes 1 byte 2 of a subcode and 32 bytes of audio channel data, 24 bytes of 3 which are allocated for L and R audio channel data (each channel 4 having 6 samples with 2 bytes per sample) and 8 bytes allocated to an error correction code. The frames are grouped as a block 6 of 98 frames (Frame 0, Frame 1, ..., Frame 96 and Frame 97) as 7 shown in Fig. 18B and eight of these blocks P,Q,R,S,T',U,V and W
8 are transmitted as shown in Fig. ~18C. "~'he subcodes for Frames 0 9 and ~. in each block are reserved far syznc patterns S0, S1, whereas the subcodes for the remaining 96 frames are reserved for 11 various subcode data. The first two blocks P, Q are allocated to 12 search data employed for searching through record tracks, while 13 the remaining 6 blocks R,S,T,U,V and W are available for graphic 14 data.
CD-G transmits each black of 98 frames at a repeating 16 frequency of 75 Hz. Thus, the data transmission rate for 1 block 17 is (75 Hz x 98 bytes) "7.35 kHz, resulting in a subcode bit rate 18 of 7.35 K bytes/s.
19 The transmission format for transmitting the information present in blocks R,S,T,U,V and W is shown in Fig.
21 19, wherein each of the 96 frames (2,3,... 97) of the 6 blocks 22 (R,S,T,U,V and W) is transmitted as a gasket including 6 channels 23 (R to W) of 96 symbols per channel. The packet is further 24 subdivided into 4 packs of 24 symbols apiece (symbol 0 to symbol sony.l5\3582.app PATENT

1 23), with each pack storing a CD-G character. It will be 2 appreciated that, a CD-G character is made up of 6 x 12 pixels 3 and, therefore, is easily accommodated in each 6 x 24 pack.
4 According to the CD-G format, the 6 x 12 CD--G character is stored in the six channels of (R,S,T,LJ,V and hT) at symbols 8 to 19 (12 6 symbols). The remainder of the symbols in each of the packs ? store information about the character.
8 Mode information is one example of information stored 9 in the packs and is stored in the first 3 channels (P., S, T) of symbol 0 ire each pack. Item information is another example which 11 is stored in the last 3 channels (U, V, T~) of symbol 0. A
12 combination of the mode information and the item information 13 defines the mode for the characters stored in the corresponding 14 pack as follows;
Table 1 16 Mode Item 1? 000 000 0 mode 18 001 000 graph~.cs mode 19 001 .001 TV-graphics mode 111 000 user's mode 21 An instruction is anothe:xw example of information stored 22 in the packs and is stored in all of the channels of symbol 1.
23 Corresponding mode, item, parity or additional information for 24 the instruction is stored in all Uf the r.:hannels of symbols 2 to sony.l5\35d2.app PATENT

1 7. Parity information for al:G. of the data in the channels of 2 symbols 0 to l9.is stored in all of the channels of t:he last 4, 3 symbols (symbols 20 to 23) of each pack.
4 As discussed, the CD-G system is slow. The CD-G data is transmitted at a repeating frequency of 75 Hz and, therefore, 6 a packet which contains 4 packs is transmitted at a rate of 300 7 packs per second (75 Hz x 4 packs). That is, with 1 character 8 allocated to the range of 6 x 12 pixels, 300 characters can be 9 transmitted in 1 second. However, a CD-~G screen is defzned as 288 horizontal picture elements x 192 CD-G vertical picture 11 elements and requires more: than twice the 300 characters 12 transmitted in 1 second. The total transmission time for a 288 x 13 192 screen is 2.56 seconds as shown by the following equation:
14 (288/6) x (7.92/12) . 300 = 2.56 seconds With the CD-G system, searching for a specific event 16 (such as a scene) would be extremely time consuming because the 17 time to regenerate each screen (2.56 seconds) by itself is 18 extremely long, when it is considered that screens are usually 19 refreshed in tenths. of a second. Tha.s problem is compounded when hexadecimal codes are used for the characaex°s because each 21 hexadecimal expression requires 4 bits to represent 1 pixel. As 22 a result, 4 times the data described above is transmitted, 23 thereby increasing the transmiss:i.c:an rage to 10.24 seconds (4 x 24 2.56 seconds). Since each screen requ~.res a sluggish 10.24 sony.l5\3582.app 4 PATENT

1 seconds fox transmission, a continual transmission of screens 2 means that a lag time of 10.24 seconds is experienced when 3 transmitting screens using the: CD-~G technique.
4 Tn one type of system (known as the CAPTAIN system), dot patterns, as well as character codes, represent the 6 subtitles. This system, however, does not appear to be 7 significantly better than the CD-U system a,nd suffers from some 8 of the same disadvantages. That is, both systems lack the 9 capability to search for a specific event efficiently. In addition, these systems do not provide subtitles with sufficient 11 resolution power in displaying the subtitles. The CD--G system 12 designates only 1 bit for each pixel, and~thi.s bi.nary,pixel data 13 creates undesired abasing and flicker. The CAPTAIN system, for 14 example, is developed for a 248 (horizontal picture elements) by 192 (vertical picture elements) di;~play,-~~..~., a low resolution 16 display, and not for high resolution video ~.>ictures of 720 x 480.

18 OBJECTS OF THE TNVENTION .A"NJ~ SUMMA~tY OF THE INVENTION
19 An objective of the invention, therefore, is to provide an encoding method and apparatus for enccading subtitles to be 21 played back exclusively during ?the trick playback modep i..e., 22 during fast, slow or reverse playback modes.
sony.15~3582.app PATENT

1 Another object of the invention is to provide a 2 computer-readable memory for directing a computer to search the 3 subtitles stored in the memory.
4 An even further object ref the invention is to provide a decoding method and apparatus for decoding the subtitles to be 6 displayed during the trick playback mode.
7 In accordance with the above objectives, the present 8 invention provides an encoding method and apparatus which encodes 9 a first subtitle to be displayed during the normal playback mode and writes a corresponding first address to the record medium in 11 an area reserved for such addresses. A second subtitle to be 12 displayed in a respective video frame in place of the first 13 subtitle is encoded onto the record medium at a corresponding 14 address in the area reserved for such addresses.
The present invention further provides a computer-16 readable memory for directing a comguter to playback a subtitle 17 to be displayed during a trick playback ~~ode by pointing the 18 computer to addresses of subtitles to be displayed in the trick 19 playback mode.
The present invention further provides a decoding 21 method and apparatus which decodes the s~.xbti.tles to be displayed 22 exclusively during~the trick playback mode by reading out the 23 subtitle from a subtitle address stored on the record medium.

sony.l5\3582.app ~l~~.e~iz.~
PATENT

1 BRIEF DESCRIPTION OF T~i~'~, DRAWINGS
2 A more complete appreciation of the present invention 3 and many of its attendant advantages will be readily obtained by 4 reference to the following detailed description considered in connection with the accompanying drawings, in which;
6 Fig. 1 is a block diagram of a data decoding apparatus 7 of the present invention;
8 Fig» 2 is a block diagram of the subtitle decoder 9 depicted in Fig. l;
Figs. 3A and 3B are tables of addresses according to 11 the present invention;
12 Figs. 4A and 4B are diag~~ams depi~aing subtitle search 13 operation in normal and trick playback mode; .
14 Fig. 5 is a table of communicatiozas between the system controller of Fig. 1 and the controller of Fig. 2;
16 Fig. 6 is a table of parameters for the camm.unications 1? between components of Fig. 1 and Fig. 2p 18 Figs. ?A to '7C are signal diagrams demonstrating data 19 encoding of the present invention;
Fig. 8 is a color look up table referred to when 21 encoding subtitle data;
22 Fig. 9 is a bloc~C diagram of tkae encoding apparatus of 23 the present invention;
sony.~5\3582.app PATENT

1 Figs. l0A and 10B depict a block diagram for the wipe 2 data sampler of.Fig. 9;
3 b''ig. 11 is a color look up table referred to when 4 conducting a color wipe operation;
Fig. 12 is a graph for the explanation of a code buffer 6 operation;
7 Fig. 13 is a block diagram describing the internal 8 operation of the code buffer in Fig. 2;
9 Figs. 14A to 14C depict a scheme for the colorwiping operation;
13 Fig. 15 is a block diagram depicting the colorwiping 12 operation according to Figs 14A to 14(':;
13 Figs. 16A to 16c~ depict a scheme for the dynamic 14 positioning operation;
Fig. 17 is a block diagram depicting a circuit for the 16 dynamic positioning operation according to Figs. 16A to 16C;
17 Figs. 18A to 18C depict the arrangement of data 18 according to a CD-G format.; and 19 Fig. 19 depicts a transmission format of the data in the CD-G format.
21 DETAILED DESCRIPTION OF TH PR F'ERR D EMBOD MENTS
22 Referring now to the drawingsr wherein like reference 23 numerals designate identical or corresponding parts throughout, 24 the present invention will bra desc:r:ibed .
soay.15~3582.app ~18~~~~
PATENT

1 D~:cr~d3.~ng ,Agparatus 2 The data decoding apparatus shown in Fig. 1 3 incorporates the present invention and operates to decode a 4 reproduced signal. A system controller 14 of the data decoding apparatus causes the signal reproduced from, for example, a disk 6 91, to be processed and sent to a subtitle decoder 7. The system 7 controller communicates with a controller 35 (Fig. 2~ of the 8 subtitle decoder to decode the subtitles arid superimpose them 9 onto a decoded video image for display c9n a television screen.
A data decoder and demultiplexer ~. receives a digital 11 signal reproduced from, far example, a disk, a VCR, o:~- the like.
12 The data decoder and demultxplexex~ error decades the reproduced 13 signal, preferably employing an Error ~'arrecting Cade.(ECC) 14 technir.~ue, and demultiplexes the error decoded reproduced signal into video, subtitle and audio data streams. A memory 2 may be 16 used, for example, as a buffer memory as a work area for the 17 purpose of error decoding and demultiplexing the reprc>duced 18 signal.
19 A video decoder 3 decodes the demultiplexed video data from a video data stream, A memory 4 may beg amployed for the 21 operation of decoding 'the video data sim:i.lax.- to the operation o~
22 the memory 2 employed with data decoder and demultiplexer 1.
23 A letter box circuit 5 converts t~a.e decoded video data 24 with a 4:3 aspect ratio to a 16;9 aspect ratio. The conversion sony,l5\~582,app 9 PATENT

1 is performed using a 4 to 3 decimation process, whereby every 2 four horizontal lines are decimated to three horizontal lines, 3 thus squeezing the video picture into a ,~ picture. According to 4 the letter box format, a vertical resolution component is derived from the remaining 1,~ of the video picture and is employed to 6 enhance the vertical resolution of the decimated video picture.
7 A timing adjustment memory 6 times the transmission of the video 8 picture to ensure that the ~,1 of the letter box picture is not 9 transmitted. When the decoded video data generated by the video decoder 3 is already in a 16:9 letter bar format, the letter box 11 circuit bypasses the decimation operation and sends tl~ze decoded 12 video data directly to subtitle decoder 7w 13 The decoded subtitle data demultiplexed by trhe data 14 decoder and demultiplexer Z is sent directly to subtitle decoder 7 which decades the subtitle data accorda.ng to instructions from 16 system controller 14 and mixes the decoded subtitle data with the 17 decoded video data.
18 A composite encoder 8 encodes the mixed subtitle data 19 and video data into.a suitable video picture format, such as NTSC, PAL or the like. A mode display 9 interfaces with a user 21 and indicates, for examgle, the made of a television monitor 22 connected to the illustrated apparatus, A ~?/A converter 10 23 converts the encoded signal received from the composite encoder sony.l5\3582.app l.l~

~~.~1~~-~
PATENT

1 into an analog signal suitable for disglay in the indicated mode, 2 such as NTSC or. PAL.
3 The audio portion of the audio/video signal. decoded by 4 the data decoder and demultiplexer 1 is decoded by are audio decoder 11 which decodes the demultiplexed audio data. using a 6 memory 12, for example. The decoded audio data output :from the 7 audio decoder is converted into an analog audio signal 8 appropriate for reproduction by a television monitor by a D/A
9 converter 13.
Subtitle Decoder 11 Subtitle decoder '~, as will be discussed with reference 12 to Fig. 2, decodes the encoded subtitle data and mixes the 13 decided subtitle data with the appropriate video data.
14 Controller 35 (Fig. 2) controls the operations of the subtitle decoder and communicates with the system~controller 14 of the 16 decoder (Fig. 1) using the command signals shown in Fig. 2 (as 17 listed in Fig. 5). Together, the controller 35 and system 18 controller 14 time the decoding of the subtitle data so that the 19 subtitle data is mixed with video image data at the proper position whereat the subtitles are to appear on the video image.
21 A word detector 20 of the subtitle decoder receives the 22 subtitle data in groups of bit streams reproduced from a disk, 23 the bit streams being stored on the disk ire packets. Each group 24 of bit streams makes up one frame Cor page) of subtitles to be sony.l5\3582.app PATENT

1 superimposed on a video image. Different groups of bit streams 2 may represent subtitles displayed in different playback modes, 3 such as normal playback, fast-reverse or fast-forward, 4 alternatively referred tc~ as trick modes. The system controller indicates to the word detector using a stream select signal which 6 playback mode is to be adopted for display and the word detector 7 selects the appropriate bit stream of signals for the indicated 8 playback mode. In the case where different. video images are 9 displayed on different channels, the system. controller indicates the appropriate channel to the ward detector correspondingly in a 11 ch_select signal and the word detector changes channels to 12 receive only those bit streams on the selected channel.
13 A group of bit streams making up one frame and received 14 by the word detector includes page header information (s. header) which describes the format of the group of frit streams. The page 16 header information is accompanied by header error information 17 (header error) and data error information (data error), The 18 system controller uses the page header information to determine 19 how to parse the grpup of bit streams and e~ctract the relevant 2o subtitle data therefrom. The system controller uses the header 21 error information to correct anomalies i~r the page header 22 information and uses the data error. information to correct 23 anomalies in the subtitle data.
sony.l5\3582.app ~-2 PATENT

1 The word detector forwards the subtitle data. (Bitnnap) 2 along with other decoded i.nformat:~.on (:i.r~cluding a presentation 3 time stamp PTS, position data positi.an~"data and color look up . 4 table data CLUT_data) to a code buffer 22. The PTS is a signal that indicates the precise time when the audio, video and 6 subtitle data for a frame is transmitted so that the system 7 controller knows when to demultiplex the data from the reproduced 8 signal. The position data indicates the horizontal alld vertical 9 position where the subtitles are to be superimposed an the video image. The CLUT_data indicates which colors are to be used for 11 the pixels making up the subtitles. For example, the system 12 controller 14 determines that a video image is being displayed 13 and sends the subtitle data to subtitle decoder 7 at the time 14 indicated by the time stamp (FT'S) and cause.a the subtitle decoder to output tree corresponding subtitle data (~3itmap) at a position 16 in the video image represented by the horizcPntal and vertical 17 position indicated by the p~rsitiorr,data :gin the color indicated by 18 the CLUT"_data .
19 The word detector also detects display time data and forwards this data t.o the system controller (Fig. 1) v.ia, 21 multiplier 350 of controller 35 (Fig. Z). This display time data 22 is read from the page header information of a group of bit 23 streams comprising one page (i.e., a frame) of subtitle data.
24 The display time data indicates a t~.me duration in which the sony.l5\3582.app ~ 3 ~~~~'~'-PATENT

1 subtitle encoded in the page is to be displayed with a respective 2 video frame during a normal playback mode. 4~hen, for example, 3 the system controller (Fig. 1) indicates with the special signal 4 to controller 35 that the system is in normal playback mode, the controller causes the multiplier to be bypassed and the display 6 time data is forwarded directly to the system controller. In the 7 case where the special signal indicates a special (trick 8 playback) mode, the controller causes the display time data to be 9 switched to the multiplier and tine display t~.r~ae data is factored l0 before being forwarded to the system controller. It will be 11 appreciated that th.e display times stored with the tr:i.ck mode 12 subtitles are for the normal. playback mode because it is unknown 13 in advance which playback speed the va.ewer will select and that 14 these display times are modified by the cc~ntraller for different playback speeds.
16 By factoring the display time data, the time duration 17 for display of the subtitle in a trick p~.ayback mode i.s varied.
18 This is particularly advantageous when the display time for a 19 subtitle is, for example, longer than the time duration of display for a video frame in trick playback mode, In that 21 situation, displaying'the subtitle for a duration equal to the 22 unmodified display~time would undesirably result in the subtitle 23 being displayed in the trick playback mode for a longer duration 24 than the respective video Frame is c~ispl~~yed:. Tn other sony.l5\3582.app PATENT

1 situations, it may be desirable to extend the duration of 2 subtitle display beyond a single video frame when, for example, a 3 subtitle to be displayed exclusively in the trick playback mode 4 relays information to a viewer about a plurality,of the video frames. Accordingly, the multiplier factors the display time 6 data to display the subtitle for a specified duration determined 7 by controller 35 depending upon the situation.
8 A scheduler 21 is provided to ensure that the data 9 received by the code buffer 22 from the demultiglexer 1 (Fig. 1) does not overflow the code buffer. The scheduler controls 11 read/write access to the code buffer by determining the bandwidth 12 for an I/O port (not shown) which receives the bit streams 13 selected by the word detector. The bandwi~i~:.h refers to the 14 read/write rate and is calculated by dividing the rate at which the demultiplexer demultiplexes data by the number of parallel 16 bits written or read from the code buffer. For example, a data 17 rate from the demultiplexer of 20 Mbps d~.v.~ded by 8 paz-allel bits 18 results in a 2.5 Mbps rate of data read from the code buffer.
19 Therefore, the scheduler will set the read/write rate of the I/O
port in order to maintain a consistent flow rate of data into and 21 out of the code buffer. The code buffer, thus, receives the 22 subtitle data (Hitniap) and awaits a decode start signal from the 23 system controller to read c>ut the data.
sony.l5\3582.app ~1~1~~~t PATENT

1 Advantageously, the system controller executes reading 2 in real time when it is determined from the horizontal and 3 vertical sync signals that t'he television display is at a 4 position corresponding to the position indicated by the position data. For real time display, the reading rate should correspond 6 to a picture element sampling rate, preferably 13.5 MHz. As 7 discussed, the subtitle data preferably is written into the code 8 buffer at a rate of 2.5 MHz or more. Thus, the 13.5 MHz sampling 9 clock is divided into four clock cycles of 3.375 MHz each. Qne of these 3.375 MHz clock cycles is allocated to writing (because 11 writing requires at least 2.5 MIHz) and the remaining three clock 12 cycles are allocated to reading data from~the code buffer, thus 13 satisfying the requirement for real time display.
14 The read/write operation described is not only advantageously performed i,n real time, but also provides high 16 resolution. Eight bits of the subtitle data are read from the 17 code buffer 22 for each of three read clock cycles, or twenty-18 four bits per sampling clock. When display of the picture is 19 conducted by the television monitor every fourth clock cycle, one-fourth of the twenty-four bits, X24/4 .-) 6 bits are displayed 21 at every clock cycle. That. is, each sub~::it~e picture element may 22 comprise six bits, which is more than sufficient to achieve a 23 high quality of resolution for the subti.ti.es.
sony.l5\3582.app PATENT
4.50100-3582 1 The operation of the code buffer 22 and corresponding 2 components of Fig. 2 is depicted in the black diagram in Fig, 13.
3 The code buffer 22-1 accumulates bit streams of subtitle data 4 until at least one page of subtitle data is accumulated in the code buffer. The subtitle data for one page is transferred from 6 the code buffer 22-1 to the display memory 22-2 (which acts as a 7 buffer for the subtitle decoder) when the subtitle portion of the 8 display time stamp (PTS) is aligned with the synchronizing clock 9 (SCR). The synchronizing clock advances a pointer in the display to memory 22-2 during reading indicating which address of the stored 11 subtitle data is being current~.y read. ~t will be noted that 12 placing the code buffer and display memory in a single unit is 13 preferred since the code buffer need only increment ore pointer 14 for pointing to the current address in the display memory 22-2 which stores the next set of subtitle data. With an internal 16 memory, therefore, virtually no delay is attributed to a 17 transfer operation, resulting in a high speed transfer of the 18 subtitle data.
19 When the code buffer is read during a normal playback mode, the synchronizing clcack advances trm pointer of the display 21 memory 22-2 at each clock pulse. However, during special (or 22 trick) reproduction (such as fast-forward, fast-reverse playback 23 modes), the pointer is advanced at a different rate. To this 24 end, a special command is first sent to the controller 35 and the sony.15~3582.app 9~ ~ ~.ø
PATENT

1 controller sends back an acknowledge signal. ~sgecial-xaek), 2 acknowledging that special reproduction is to be initiated. To 3 uniformly speed up (or slow down) the operations of the subtitle 4 decoder according to the special reproducta.on race, the system clock reference (SCR3 can be altered by adding or subtracting 6 clock pulses. Subtraction pulses are created at an n times rate 7 corresponding to the rate of fast-feeding c>r fast-reverse 8 feeding. For example, at the time when special reproduction is 9 commenced, real time subtraction is performed on the bit stream of subtitle data read out from the code buffer at the n times 11 rate and the pointer advances at the desired rate to effect the 12 special playback mode.
13 When the special reproduction operation corresgonds to 14 a pause operation, on the other hand, no subtraction pulses are created. Instead, an identical frame is-continuously read from 16 the code buffer repeatedly, thus providing the illusion sensation 17 that the subtitles are paused.
18 The reading operation is ended when subtitle decoder 7 19 determines that an end of gage ~EOP) of the subtitle frame is reached. The system controller sends a repeat time signal to the 21 controller 35 which indicates the length off: a page. A.n inverse 22 run-length circuit~24 includes a counter and sends a display end 23 signal to the controller 35 when the count value of the counter 24 reaches the value indicated by the repeat time signal. When sony.l5\3582.app 1 ~

PATENT

1 controller 35 determines that the repeat time is reached, the 2 reading operation of the code buffer i.s stopped. For purposes of 3 this invention, the code buffer preferably stores at least two 4 pages of subtitle data because one page will be read while another page is written into the code buffer.
6 Controller 35 issues a buffer overflow signal to system 7 controller 14 when an overflow of code buffer 22 occurs. An 8 overflow can be determined when the controller receives the 9 display end signal from inverse run-length circuit 24 before word detector 20 receives an end of page iEOP) signal on the following 11 page. At that time, the system controller withholds transfer of 12 subtitle data from data decoder and demultiplexer 1 (~''ig. 1) to 13 the word detector to prevent an overflow of the code buffer.
14 When an overflow condition has passed, the next stream will be written into the code buffer and displayed at the correct display 16 start position.
17 An underflow condition exists when code buffer 22 has 18 completed reading the subtitle data for an entire page and no 19 further data exists in the code buffer. The code buffer is depicted with a capacity of two pages by the "code buffer size"
21 line in Fig. 12. Graphically, an underf.~.ow would appear in Fig.
22 12 as one of the vertical portions of ~.ine ~C) which extends 23 below the lower limit of the c:ade buffer. Ey contrast, an 24 overflow condition is graphically depicted in Fig. 12 when the a._ 9 sony.l5\3582.app PATENT

1 subtitle data read into the code buffer is too large, i.e., the 2 horizontal portion of line (C) extends beyond line (B).
3 F'ig. 12 graphically demonstrates the data flow into and 4 out of code buffer 22. The T-axis (abscissa) represents time, while the D-axis (ordinate? represents data size for each page of 6 data. Thus, the gradient (rise/run) represents the data flow 7 rate of the subtitles into the code buffer. Graph (C) represents 8 the data flow of the subtitle data. The vertical portions of 9 graph (C) indicate a transfer of subtitle data from the code buffer when the display time stamp (PTS) is aligned with the 11 synchronizing clock (SCR) generated internally by subtitle 12 decoder 7. The horizontal portions of the graph (C) indicate the 13 transfer of subtitle data into the code buffer, the display time 14 for normal playback made indicated by the horizontal. lines with arrows. For example, at a time that the-display time stamp (PTS) 16 for page (SO) is received by the code buffer, the previous page 17 of subtitle data is transferred from the code buffer and page 18 (SO) is written into the code buffer. then another display time 19 stamp (PTS) is received by the code buffer, the subtitle data of page (SO) is transferred out of the code buffer and page (S1) is 21 written in. Similarly, the remaining pages (S2), (S3) are 22 written into and read out of the code buffer as indicated.
23 To precisely time the r.~eada,ng caf the subtitle data from 24 the code buffer with the d.is~alay of the video image, delay sony.l5\3582.app ~1~~.~'~~)v'~
PATENT

1 compensation must be performed to allow for delays within the 2 subtitle decoder. This is especially important where an external 3 memory is employed as the display memory because an external 4 memory increases the delay factor. Delay compens.atiori is achieved by controlling the timing of the decode start command 6 from system controller 14. The system controller delays the 7 decode start command by a time equal to the processing of a 8 letter box picture (approximately one field) and a delay caused 9 by video decoding at the a.nstar~t the synchronizing clock of the controller (SCR) is aligned with the display time stamp (PTS).
11 Delay compensation is particularly useful,, since the video, audio 12 and subtitle data are multiplexed on the premise that the decode 13 delay in each of the video, audio and subtitle data signals is 14 zero in the data encoding apparatus.
When the subtitle data for one-page is read out of the 16 display memory 22-2 (~':ig. 13~, the headers c~f the bit streams are 17 separated therefrom by a parser 22--3 and the remaining data .is 18 forwarded to the inverse variable-length eider or run-length 19 decoder 23, 24 during a vertical blankixxg period (V) . Inverse VLC (Variable Length Coding) circuit 23 ~'F~g. 2) subjects the 21 subtitle data to variable lerxgth decoding. The variable length 22 decoded subtitle data i.s c~amposed of le~rel data ("1'° car "0'° ) and 23 - run data as paired data. :In the case where variable length 24 decoding is not employed, the inverse VLC circuit may be bypassed sony.15~3582.app ~1~I~!~
PATENT

1 and the subtitle data read from the code buffer will be directly 2 output to inverse run-length circuit 24. Inverse run-length 3 circuit 24 conducts run-length decoding by generating the level 4 of data from the number of run data elements. Thus, VLC circuit 23 and run-length circuit 24 decompress the subtitle data which 6 had been stored as compressed data in code buffer 22.
7 The decompressed subtitle data is then sent to a 3:4 8 filter 25. The 3:4 filter receives an xsc,~ueexe signal from the 9 system controller 14 indicating the aspect ratio of the corresponding television monitor. Where the signal indicates 11 that the monitor has a 4:3 aspect ratio, the 3:4 filter applies 12 3:4 filtration processes to the subtitle data to match the size 13 of the subtitles to the size of the video picture. Tn the 14 preferred embodiment, the controller 35 reads 90 pixels worth of subtitle data from the code buffer 22 before the H sync pulse is 16 generated. Tn the case where the television monitor already has 17 a 16:9 aspect ratio, or the decompressed subtitle data represents 18 fonts, the 3:4 filter is bypassed.
19 A color look-up table 26 (CLUTa r~~sc:eives the: subtitle data from the 3:4 filter 25 and the CLTJ"~ data from the code 21 buffer 22. The color look up table generates a suitable color 22 from the CLUT_data for the subtitle data. '~'he color look up 23 table selectr.s an address corresponding tca ~,.he subtitle data for 24 each pixel a.nd forwards a mixing ratio K and color components Y
sony,l5\3582.app PATENT

1 (luminance), CR (color difference signal F-Y) and Ca (color 2 difference signal B-Y) to the mixer 34* The color components Y, 3 CR and CB, when mixed by mixer 34, at the mixing ratio K create a 4 pixel with the color indicated by the valor look up table.
Background video data is zncot~porated in the 6 arrangement of the valor look-up table. For example, address 0 7 of the look-up table includes key data K having the value of 00 S h; which means that the subtitle data will not be seen and the 9 background video data will manifest, as shown by regions T1 and T5 in Fig. 7c. Addresses lh to 6h of the look-up table include 11 values of the key data K which increase linearly (20, 40 ... CO
12 hexadecimal); which means that the subtitle pixels according to 13 these addresses are mixed with the background data as shown by 14 the regions T2 and T4 in Fig. 7c. Finally, addresses 3h to Fh of the look-up table include values of key data K of EOh; which 16 means that the components Y, Cr and Cb are mixed without any 17 background video data as shown by region T3 in Fig. 7c. The 18 color look-up table data is generated from the system controller 19 and is previously downloaded to ~:he CLUT circuit before decoding.
With the color look-up table, the filtered subtitle data is 21 transformed into the appropriate color pixel for display on the 22 television monitor.
23 Fig. 8 shows one example of a color look-up table where 24 the components Y, Cr, Cb and K axe arranged according to the sony.l5\3582.app PATENT

1 addresses O...F (hexadecimal). As will be explained, color 2 wiping is performed by changing the CLUT_data, thereby replacing 3 part of the color look up table by the color wiping color look up 4 table shown in Fig. 11. Normally, a particular subtitle frame is refreshed several times because frames are refreshed in a 6 television signal several times a second. When the subtitles are 7 refreshed, the same subtitle data will be employed. However, the 8 color will be different due to the chanced color look up table.
9 Thus, the subtitles will appear to be colox° wiped as they are refreshed with each. consecutive frame.
11 A. mixer 34 (Fig.. 2) mixes the pixels from the color 12 look-up table 26 with video data from video decoder 3 (Fig. 1).
13 The resulting mixed data represents a video picture with 14 superimposed subtitles and is ready to be o~xtput to a television monitor. Mixer 34 is controlled to position the subtitles within 16 the video picture by referencing a u~pos3.ti.on signal generated by 17 system controller 14 from commands of an operator via controller 18 35. The a position value designates the vextical position for 19 display on the screen and may be varied (either by a user, the transmitter, or otherwise) allowing a user to place the subtitles 21 anywhere along a vertical axis.
22 The decoding apparatus of the present invention may be 23 practiced with the parameters for the d~.~fexerit sa.gnals shown in 24 Fig. 6. However, the present inve~ation i.s z~ot limited to the sony.l5\3582.app 2 4 a PATENT

1 parameters set forth in that figure and may be employed in 2 different video systems, 3 With the present invention, a user has cantrol over the 4 display of the subtitle through a made display device 9 (Fig. 1).
System controller 14, upon cammancf from the user, sends a control 6 signal to mixer 34 (Fig. 2), turning the subtitles on or off, 7 Since the present invention decodes subtitles in real time, the 8 user does not experience any unpleasant delay when turning the 9 subtitles on or off. Tn addition, the subtitles can be controlled, by the user or_ otherwise, t.o fade-in/fade~-out at a 11 variable rate, This is achieved by multiplying a fadE>
12 coefficient to the pattern data representing the subtitles at a 13 designated speed. This function also allows an editor of the 14 subtitles to present viewers with different sensatians according to the broadcast of the audio/video pict~zre. For example, news 16 information may be "flashed" rapid.~y to draw the attention of the 17 viewer, whereas subtitles in a slow music video "softly" appear 18 in order not to detract from the enjoyment of the music video, 19 - Subtl,tle Searching The present ~.nvent:~.on provides a viewer with the 21 capability to search for a specific scene or sound quickly and 22 efficiently, For example, the: viewer may wish to quickly locate 23 a specific dialogue in a video recardirng of an office meeting 24 without watching lengthy periods of relative inactivity in the sony.l5\3582.app k PATENT

1 meeting. The present inventic~an pr~avides subtitles exclusively 2 displayed during the trick playback mode and the viewer can 3 quickly locate the desired dialogue by scanning such subtitles 4 during the trick playback mode. Preferably, only those video frames which have subtitles displayed in the trick playback mode 6 are played back, skipping over the scenes of relative inactivity 7 to the important scenes. The subtitles for display in the trick 8 playback mode may be narrative in that a si,.ngle page of subtitles 9 corresponding to one video frame may be displayed over the course of a plurality of video frames to descr~..be an entire scene or 11 dialogue. In this manner;, the v~.,ewer can easily loca~:e any 12 information in the video picture desired quickly arid efficiently 13 and need not wait through periods of relative inactivity.
14 An example of displaying subtitles from fait streams of data decoded from the record medium is shown in Figs. 4A and B, 16 wherein the bit streams include video bi~.t streams (V I, V P, 17 V B), audio bit streams (A) and subtitle bit streams (Sp, Sw).
18 Reproduction of the subtitle (Sp) for norma:~ playback mode is 19 shown in Fig. 4A, where it will be noticed that the subtitle (Sp) for the normal playback mode is divided intro three bit streams 21 which must be recombined to comprise one page of subtitles for a 22 respective video frame. When the tape :i.:~ moving in a forward 23 direction in the normal playback mode, the bit streams of the 24 subtitle (Sp) are reproduced by causing t.~~e pickup to ,dump to the sony.15~~582.app 2 6 I.
PATENT

1 individual bit streams of the subtitle as indicated by the arrows 2 extending from left to r:i.ght i.n Fig. 4A. Similarly, when the 3 tape is mowing in a reverse direction in the normal playback 4 mode, the bit streams of the subtitle (Sp) are reproduced by causing the pickup to jump to the bit streams according to the 6 arrows extending from right to i.eft. After the bit streams for 7 the subtitle (Sp) are reproduced, the decoder of Fig. 2 decodes 8 the subtitle (Sp) for display with the corresponding video frame.
9 According to th.e present invention, subtitles (Sw) to be displayed in the trick playback. made, which may be different 11 in textual content from the subtitle (Sp), are recorded adjacent 12 to video frames (V I). It will be appreciated that not all video 13 frames (V I) will be displayed in the tx°ick playback mode and do 14 not have an adjacent subtitle as shown ~..n Fig. 4B. T:he subtitle decoder of the present invention, during.a forward trick playback 16 mode, will decode each subtitle (Sw) in the direction of the 17 arrows from left to right in Fig. 4B and, similarly, decode in 18 the direction of the arrows from ~c°ight: to left in reverse trick 19 playback mode. Notably, t:he t~ni.rd video frame (V r) is skipped because that video frame does riot have a subtitle (Sw) since, for 21 example, that video frame has little activity.
22 With the'present invention, the subtitles displayed in 23 trick playback mode may be of different textual content from the 24 normal playback subtit:Les. This means that the trick mode sony.l5\3582.app PATENT

1 subtitles may include "special" information describing the scene 2 or corresponding sounds. Indeed, the trick mode subtitles may be 3 played back without the normal playback subtitles or the video 4 picture, which provides the viewer with the ability t:o quickly find the desired information by scanning only the trick mode 6 subtitles. Thus, the present invention provides quick and 7 efficient location of information in a video picture by skipping 8 to those video frames which have significaxat information.
9 The present invention, during encoding, stores a starting sector address where the page header portion of the 11 subtitle bit streams are located in areas reserved for addresses 12 on the record medium for later retrieval during decoding. In a 13 first embodiment, the invention stores the addresses of the 14 subtitles to be displayed in trick pla~rback made in a table of contents (TOC) (Figs . 3A, B) lc:~cated at. the beginning of the 16 disc. The table of contents i:a not encoded and, therefore, is 17 read directly by the system controller to drive the pickup to the 18 addresses corresponding to the subtitles to be displayed in trick 19 playback mode.
The table of contents shown in Fig. 3A identifies 21 frames by f tame number' ( subcode f tame # ) and pointers ( POINT ) 22 pointing to a track on the disk at which the corresponding frame 23 is located, and time codes (PM3_N, PBEC, P~'T~2AME) corresponding to 24 that frame . The table of contents ident:~fa.es pages of stream sony.l5\3582.app 2 CA~02181864 1996-08-28 t~yY~ "p~. n"., i. t..,-. v,,~r ..w..
PATENT

1 data by indicating the frame, starting sector address, and ending 2 sector address of recorded pages. From this table of contents, 3 the system controller can. locate the beginning of the subtitle 4 pages and cause drive control 15 (Fig. 1) to jump the pickup to the sector indicated by the start~s~ector address in the table of 6 contents.
7 As shown in Fig. 3B the present invention may also 8 store in the "trick playback dux°at:ion" ~~olumn, the display time 9 data for the subtitles to be displayed. The controller (Fig. 2) determines the duration of display of the subtitle for various 11 playback speeds using display time data by changing the 12 multiplicative factor of multiplier 350 which factors the display 13 time data in proportion to the playback speed.
14 In a second embodiment of the invention, the sector addresses for the page header portions are collected in an area 16 of the disk called a stream map. Similar to the previous 17 embodiment, the addresses here are directly employed say the 18 system controller (Fig. 1) without the need for decoding. Unlike 19 the table of contents, however, the stream map is not necessarily confined to the beginning of the disk, bo.zt may be located at. any 21 sector. The stream m~.p is, thus, arranged as a packet with 22 video, audio, subtitle, blanking, packet length of stream, 23 identifier, and length of stream map information. The system 24 controller references the stx°eam map in a similar manrnur to the sony.l5\3582.app 2 9 !~.$1~~
PATENT

1 table of contents, thus causing the pickup to reproduce pages of 2 subtitles by sending the reproduced pages of subti,tlE~ streams to 3 the subtitle decoder.
4 In a third embodiment, the sector addresses of the previous and following pages ~,f subtitle streams are stored in 6 each page header portion of a currently reproduced page. Since 7 the sector addresses are in the page header portions, the sector 8 addresses are sent to the subtitle decoder to he deeded, not the 9 system controller. The word detector 20 of the subtitle decoder (Fig. 2) detects the subtitle stream sector. addresses (subtitle 11 stream sector address) and forwards them to the system 12 controller, via controller 35. As each subtitle is decoded in, 13 for example, the forward playback mode, the system controller 14 recalls the following sector address for the next page and causes the pickup to jump to the sector indicated by that following 16 address to reproduce the next page caf sz~btitles. Sim:i.larly, in 17 the reverse playback mode,, the system controller recalls the 18 previous sector address for the pr°evious page and causes the 19 previous page of subtitles to be reproduced. Specifically, the word detector detects whether a stream includes sector. addresses 21 according to the following operation:

sony.15~3582.app ~ ~) ~ll~it PATENT

1 No. of bits Mnemonic 2 userrdata~f lag ~. uimsbf 3 if(user~data~flag = "1"1[
4 lengtrl_of user~data . 16 bslbf next subtitle address offset 32 bslbf 6 reserved 8 bslbf 7 previous~subtitle~address~offset 24 bslbf 8 reserved 8 bslbf 9 l l0 System controller 14 (Fig. 1) acts as a computer and 11 executes the above operation, causing the word detector to 12 determine if the user~data~flac~ is set to "~" and, if so, treats 13 the next 16 bits as lengthp~of~user data; the next 32 bits as 14 next subtitle address offset; the next 8 bits as reserved; the next 24 bits as previous~subtitle~address~offset; and the last 8 16 bits as reserved. The word detector forwards this information to 17 the system controller, via the c~antrolle:r~ :~~:~~, and continues 18 detecting subtitle streams. The system c:orxtroller receives the 19 subtitle page sector addresses from this operation and controls the decoding apparatus as descrihed.
21 The record triedium, thus, functions as a computer-22 readable memory which directs the system con~tro~.ler as a computer 23 to playback the subtitles during a trick playback mode. The 24 addresses as shown in, for exam~ale, ~'ic~~. 3A and B, acts as sony.l5\3562.app 3 ~.

~r~..~J~..~..3~v~
PATENT

1 pointers which direct the computer (i.e., system controller 14) 2 to skip to a location on the record medium where a subtitle to be 3 displayed in the normal playback mode is recorded. During the 4 trick made playback, the pointer directs the computer to skip to a location on the .record medium where a subtitle to be displayed 6 during the trick playback mode is recorded More specifically, 7 the pointers cause the system controller t:o drive the drive 8 controller 15 to move a pickup (Fa.g. ~.) tc~ jump to the indicated 9 addresses of the record medium and read the subtitles. It will be appreciated that the computer-reada.b~.e memory may be, for 11 example, a video disc, a tape medium, or the like.
12 Encoding Technique 13 The encoding technique ernployed in the present 14 invention will be described in more particular detail with reference to Figs. 7A, 7B, 7C and F"ig. 8: As an example, the 16 technique for encoding the; letter °°A°' of F:i~:~. 7A
will be 17 explained. The letter "A" is scanned along successive horizontal 18 lines and the fill data of Fig. 78 a.s generated for the letter 19 "A" along each horizontal 'line. 'fit wi.'.1,1 be noted that the level "EO" demarks the highest level for recreating a color pixel from 21 the color look-up tabl"e shown in Fig. 6, whereas level "0"
22 represents a lack rif subtitle data.
23 The key data (K) (or mixing ratio) determines the 24 degree to which the fill data is ma.:xed wa.th background video.
sony.l5\3582.app 3 2 PATENT

1 Regions T1 and T5 of the key data correspond to areas in the 2 video picture that are not superimposed with the fil~. data;
3 therefore, these areas are designated as level 0 as indicated by 4 address 0 in Fig. 8. Regions T2 and T4 are mixed areas where the subtitles are gradually mixed with the background video picture 6 so that the subtitles blend into the ba~.kground video picture and 7 do not abruptly contrast therewith. Any of the fill data in this 8 area is stored in addresses ~. through 6 of the color look-up 9 table. The main portion of the letter "'A'" is displayed within the T3 region where the background information is muted. The 11 subtitle information in regs.on T3 is stored as addresses 7 to F
12 (hexadecimal). The color look-up table: of Fig. 8 is .arranged in 13 varying degrees of the luminance component Y. When a,pixel in 14 the region T3 is to be stared, for example, and the level of the luminance component Y for that particular pixel is 20 16 (hexadecimal), the color ~.nformation for that pixel is obtained 17 from address 9 (Fig. 8). ~:n this manner, the remaining pixels 18 for the subtitle characters are encoded.
19 . En~:odxng Apparatus The encoding apparatus of the present invention is 21 depicted in Fig. 9. Audio and video information is received by a 22 microphone 53 and video camera 51, respectively, and forwarded to 23 a multiplexes 58. The subtitle data are entered through either a 24 character generator 55 or a flying spot sc~xnner 56 and encoded by sony.lS\~582.app ~1~1 PATENT

1 a subtitle encoding circuit 5"I. The encoded subtitle information 2 is sent to multiplexer 58 and combined with the audio/video 3 information for recording onto a record disc 91 or supplied to a 4 channel for transmission, display,, recording or the .like.
video camera 51 generates the video signal and supplies 6 the same to~ a video encoding unit 52 which converts tine video 7 signal from analog to digital form. The digitized video signal 8 is then compressed for video transmission and forwarded to a rate 9 controller 52a, which controls the rate that the compressed video data is transferred to the multiplexer in synchronism with the 11 rate that the subtitles are sent to the multiplexer. In this 12 manner, the compressed video data i.s combined with they subtitle 13 data at the correct time. Similarly, audio information is 14 obtained by microphone 53 and encoded by an audio encoding unit 54 before being sent to the multiplexes.- I~ue audio encoding unit 16 does not necessarily include a rate controller because the audio 17 data may ultimately be recorded on a different track or 18 transmitted over a different channel from the video data.
19 The subtitles are generated by either character generator 55 or flying spot scanner 56. Tt°xe character generator 21 includes a monitor and a keyboard which allows an operator to 22 manually insert subtitles into a video p~.cture. 'The operator 23 edits the subtitles by typing the subtitles through the keyboard.
24 Flying spot scanner 56, on the other hand., is used for the sony,l5\3582.app ~ 4 PATENT

1 situation where subtitle: are already provided in an external 2 video picture or scanned in as text. The flying spot:, scanner 3 scans the video picture and determines where the subtitles are 4 positioned and generates corresponding subtitle data therefrom.
The subtitles from the flying spot scanner are pre-processed by a 6 processing circuit 63 to conform with subtitles generated by the 7 character generator before fort"her processing by the subtitle 8 encoding circuit.
9 The subtitle data from either character generator 55 or flying spot scanner 56 are, then, selected for compression. The 11 character generator outputs blanking data, subtitle data and key 12 data. The subtitle data and key data are~~.orwarded to a switch 13 61 which is switched according to a predetermined timing to 14 select either the subtitle or key date. The selected data from switch 61 is filtered by a filter 72 arid supplied to atnother 16 switch 62, Switch 62 switches between blanking data, the 17 filtered data from the charactQr geraers.tor, and the processed 18 data from the flying spat= scanner. When it is determined that no 19 subtitles are present, the blanking data is chosen by switch 62.
Where subtitles are present, switcta 62 chooses between the 21 character generator data c~r the flying spot scanner data, 22 depending upon which device is being used to generate the 23 subtitle data.
sony.l5\3582.app .3 5 PATENT

1 The data selected by switch 62 is quantized by a 2 quantization circuit 64, using a quantization level based on data 3 fed back from a subtitle buffering verifier 68, The quantized 4 data, which may be compressed, is supplied to a switch 69 and (during normal operation) forwarded to a differential pulse code 6 modulation (DPCM) circuit 65 for pulse cads modulation. The 7 modulated data is run-length encoded by a run-length coding 8 circuit 66, variable-Length erac4d~:.ad by a variable-length encoding 9 circuit 67 and forwarded to the subtitle buffering verifier 68 for final processing before being sent to multiplexer 58.
11 Subtitle buffering verifier 68 verifies that the 12 buffer is sufficiently filled with data without overflowing.
13 This is done by feeding a control signal. (referred to in Fig. 9 14 as a filter signal) back to the quantization circuit. The control signal changes the quantization level of the quantization 16 circuit, thereby changing the amount of data encoded :~°or a 17 particular subtitle. By increasing the quantization :Level, the 18 amount of data required far the subtitle data is reduced and the 19 bit rate of data flowing to the subtitle buffering verifier is consequently reduced. When the subtitle buffering vex°ifier 21 determines that there is an underflow of data, the control signal 22 decreases the quantization level and the amount of data output 23 from the quantization circuit increases, thereby fi.l.ling the 24 subtitle buffering verifi~:r.
sony.l5\3582.app ~ 6 ~.~V~".
PATENT
4.50100-3582 1 The subtitle buffer~.ng verifier is also responsible for 2 preparing the subtitle data fear tx~ansmiss.ion (over te:levisi.on 3 airwaves, for example). The subtitle buffering verifier, to this 4 end, inserts information necessary to decode the encoded subtitle data. This information includes a normal/special pla~r signal 6 which indicates whether the subti~.les are recorded in, a normal or 7 special (fast-forward,/reverse) mode (referred to above as the 8 trick mode). An upper limit value signal i.s inserted which 9 indicates the upper limit for the memory size of the subtitle data for a frame. An EDP signal marks the end of page for the 11 subtitle data frame and also is inserted, A time code signal is 12 inserted which is used as the time stamp PTS in decoding.
13 Subtitle encoding information is inserted and includes 14 information used in encoding the subtl.t~.~ a data, such ~~s the quantization factor. Positional information is inseri;ed and is 16 used as the position_data upon decoding. A statie/dynamic signal 17 is inserted which indicates whether the subtitle data is in 18 static or dynamic mode. The svbt.it,le buffering verifier also 19 inserts the color look up table address for transm~.ss~.on to the decoder so that the colors of the display will match the colors 21 employed in creating the subtitles.
22 The subtitle buffering verifier iu preferably a code 23 buffer similar to the code buffer of the dec~:oder (Fig.2). To 24 that end, it is useful to think of the operation of the subtitle sony.l5\3582.app PATENT

1 buffering verifier to be in symmetry (i.e., performing the 2 inverse functions of the code buffer) with the code buffer. For 3 example, the color gixels~ of tae subtitles are converted into 4 digital representations; the x°esultant digital subtitles are encoded by the run length encoder arid the variable length 6 encoder; header information is added; and the resultant subtitle 7 information is stored in a buffer and forwarded to multiplexer 58 8 for multiplexing with the audio and video data.
9 Multiplexes 58 preferably employs time-sliced multiplexing; and alsa provides error correction processing 11 (e. g., error correctian coding) and modulation processing (e. g., 12 EFM, eight-to-fourteen modulation}. The multiplexed data is then 13 transmitted (via television brc~adc~asting, recording, car ather 14 means of transference) to the decod.zng apparatus for decoding and display.
16 Encoding subtitles 17 The present .i.nve:nti.or~ permits a. viewer to search for a 18 specific scene or audible speech by providizzg subtitl~:s to be 19 exclusively displayed in the trick playback mode. The manner in which the encoder, already described with reference to Fig. 9, 21 encodes the addresses will now be discussed. After multiplexes 22 58 multiplexes the audio, uidec;~ and aubt,itlc* data streams, t:he 23 multiplexed data is sent to a sectorizing px-ocessor 100 which 24 arranges the data streams into fixed length sectors of packets.
sony.l5\3582.app -~ 8 PATENT

1 At this point, the data streams are ready for airwave 2 transmission. When the data :~txeams are to be recorded on a 3 disc, however, a table of contents (TOC) & stream map generator 4 101 determines the addresses of the pages of data streams to be recorded on the disk.
6 According to the first embodimentW, the TOC ~ stream map 7 generator generates the table of contents shown in Figs. 3A, B
8 from the sectors generated by the sectorizing processor and the 9 videoJaudio search information genex°ated, for example, by a viewer. In the second embodiment, the TOt' & stream map generator 11 generates the stream map from the sectox°s generated by the 12 sectorizing processor. Unlike tYae previous embodiment, the TOC &
13 stream map generator inserts the stream map as a packet onto the 14 disk. In the first two embodiments, system controller 14 of the data reproducer (or receiver) reads the table of contents or~ the 16 stream map directly and causes the decoding apparatus (Fig. 1) to 17 decode the streams wh:ic:h relate to the data type being searched.
18 In the third embodiment, the 'T"OC' ~ sere*~zn map generator .inserts 19 the previous and following page addresse;~ into each of the page header portions of the pages, Unlike the first two embodiments, 21 the system controller must cause the subt:.itle decoder to decode 22 the page of subtitle streams and extract therefrarn the sector 23 addresses. As described, the Tc:7~' &. stream map generator encodes 24 each stream 'with 1 bit ref a user data'ila.g that indicates whether sony.l5\3582.app 3 9 PATENT

1 stream addresses are for~hcoming in the stream; the next 16 bits 2 as length of~user~data; the next 32 bits as 3 next subtitle address offset; the next: 8 bits as reserved; the 4 next 24 bits as previous_lsubtztle~addressiaffset; and the last 8 bits as reserved.
6 According to the present invention, the video image, 7 the audio track, and the subtitles are arranged in units of pages 8 (i.e., frarnes) on the disc and the system controller accesses 9 information by recalling Pram the disc the pages of streams.
With this scheme, the system cantx~aller can cause the decoder to 11 decode only those subtitles which correspond to video frames of 12 relatively significant information.. Thus; a viewer can browse 13 through the video picture by reading the subtitles and view only 14 those portions which are of interest.
Colorwiping Encoding 16 Colorwiping refers to a pracc:~s key which an image, such 17 as the subtitles, is gradually averlaic~ with another image. An 18 exemplary application of c:olarwiping ~.~~ highl.ighting, wherein a 19 frame of subtitles is dynamically highlighted Pram left to right with the passage of time. The present. invention performs 21 colorwiping by changing the color look up table at different 22 paints of time during the subtitle display. For example, an 23 initial subt=itle frame is r~enex°ated with the standard color look 24 up table in Fig. 8. When c:o~.a~~°wi.p:irag is performed, the color sony.15~3592.app PATENT

1 look up table is changed to the color wiping look up table of 2 Fig. 11. With the passage of each frame, the gradual change of 3 the position at which the valor lock up table is changed from the 4 colorwiping to the standard color look provides t:he sensation that the subtitles are cr~angir~g color dynamically over time from 6 left to right.
7 An encoding operation for color wiping will now be 8 discussed with reference to Figs. 9 and 10. During the course of 9 encoding subtitles, an operator may desire to valor wipe the previously encoded subtitles. To that end, the operator is 11 provided with a wipe lever 81 to control the colorwiping and a 12 monitor 84 to view the color wiping in real time. The wipe lever 13 is connected to an adapter 82 to adapt t:.he analog voltages of the 14 wipe lever to digital signals suitable for digital manipulation.
The digital output of the adapter is fed~to both a switcher 83 16 and a wipe data sampler 70. The switcher sw~..tches the color look 17 up table to values represented by the position of the wipe lever 18 and generates color pixel: of the :~ubt~.tles for display on t:he 19 monitor. Thus, the'operaCor care visually inspect the colorwiping procedure while it occurs and adjust the speed ar color of the 21 wiping to satisfaction.
22 The wipe data sampler and pasi.tion sampler ?0 23 determines from the adapter signals where :i~°~ the video picture 24 the color' look up table is tc~ be changed aTW outputs this sony.l5\3582.app 4 ~~.~~.~1'1 PATENT

1 information to encoding circuits 65, 66 and 67 (via switch 69) 2 for encoding and transmission to multiplexer 58. Figs. l0A and 3 10B depict a block diagram of the operation of the wipe data and 4 position sampler. A comparatar 301 compares a present pixel signal generated by the adapter with a previous pixel. signal from 6 the adapter. This is achieved by transmitting the present pixel 7 value to input A of comparator 301 while supplying th.e previous 8 pixel value latched in a register 300 to input B of comparator 9 301. The comparatar outputs a boalean "true" value to a counter 302 (which is reset at every horizontal ox' vertical sync pulse) 11 when the present and previous pixels have the same value and, in 12 response thereto, the counter increments a count value. That is, 13 the comparator registers a true condition when the pi:~els up 14 until that point are generated Pram the same color look up table.
At the point where the color lr~ak up table changes, therefore, 16 the present and previous pixels become unegual (i.e., their color 17 changes) and the comgaratar generates a °'false" boolean 18 condition. The count value, thus, is equal to the number of 19 matches between the~present and previous values, which is the same as the position at. which the color lank up table changes.
21 The count value is latched by a. rec~zste~ 3Ct:~ upon the following 22 vertical sync pulse and transferred to the encoding circuits (via 23 switch 69) for transmission.

sony.l5\3582.app 4 y. AA lt.
.,~w°~ ~7A'~ "r~' ts' .~

PATENT

1 Colox-ariping Decoding 2 Color wiping decoding will now be discussed with 3 reference to Figs. 14A-C and 15. Fig, 3.4A shows the position 4 here the color look up table is switched at paint A from a color wiping look up table (Fig. al) to the standard color look up 6 table (Fig. 8). Fig. 14B depicts a pattern of subtitle and 7 colorwipe data arranged in discrete blocks of presentation time 8 stamps (PTS(n) ... PTS(n+t)), The firs. presentation time stamp 9 PTS(n) corresponds to normal subtit~.e data and the remaining presentation time stamps PTS(n+~. ... n+t) correspond to 11 colorwiping data (WPA ... WPZ). F'ig. 14C shows successive frames 12 (n ... n+t) which correspond to the presentation time stamps. To 13 execute colarwiping, each successive colorwiping frame (WPA ...
14 WPZ) sets the point where the color look up table is ~>witched (point A) further aloncg the displayed subtitle, thereby 16 dynamically performing calorwiping as a fuxucaian of time.
17 An operational block diagram ai° t~°ye colorwiping 18 decoding is depicted in Fig. 15. '~''he verta.cal sync pulse 19 triggers a register'205 to latch the current: subtitle frame from a display buffer (Fi.g, l5 :chows a calorw.~..pix~g frame WP being 21 latched). The colorwiping data latched xay the register indicates 22 the position of the color aaak up table switching. A pixel 23 counter 208 decrements the value indicated by the calorwiping 24 data at each horizontal syxic pu~.se and outputs a boalean "true"
sony.l5\3S82.app 4 3 ~1~~~~~
PATENT

1 flag to color look up table 26. whale the flag is "t:rue" the 2 color look up table employs the colorwiping table (Fig. 11) to 3 decode the colors of the subtitle pixels. When the pixel counter 4 reaches zero, the position of solar look up table switching is reached and the pixel counter issues a Boolean "false" flag to 6 color look up table 26. At this time, the color look up table 7 switches the colorwiping color look up t:abl.e (Fig. 11) to the 8 standard look up table (Fig. 8), and the remainder of the 9 subtitle frame is displayed in standard color. mode. Each successive colorwiping frame (WPA .,. WPB) manes the position of 11 switching; thus, each refs..~eshed subtitle frame advances (or 12 retreats) the colorwiping, thus performing dynamic ca:Lorwiping.
13 The colorwiping color look up table in Fig. 11 14 incorporates two sets of colors (one set for addresses Oh to 7h and a second set for addresses 8h to Fh)a Thus, the c:olorwiping 16 color can be changed to a secondary solar simply by changing the 17 most significant bit (MSB) of the valor look up table address.
18 For example, the first set of colas°wiping colors has a MSB of 19 "0", while the second set has a MSB of "1". Changing the MSB of address 7h to a "1" transforms the address to Fh and the 21 colorwiping valor changes. This may be done, for example, by 22 setting the MSB equal to the flag of pixel counter 208.
23 Employing the MSB to change between solar sets has the 24 advantage of reducing the number of bit: required to be encoded.
sony.l5\3582.app 4 4 PATENT

1 Since the MSB is known, only the 'three lower order bits need to 2 be encoded where 4 bits are emplo~Ted for every pixel. Where two 3 bits are employed for every pixelA the subtitle data is coded 4 only for the least significant bit. Tn a 4 bits. per 2 pixel format, only the MSB is employed for coloxT control and the 6 remaining three bits can be reserved for pa"xe1 information.
7 Thus, by using the MSB the number of bits encoded can be 8 decreased and the overall processing time for encoding and 9 decoding is optimized.
Dynamic Sut>ti.t~.e Pasitiraning 11 The subtitles are repositioned dynamically, i.e., as a 12 function of time, by employing a similar technique as described 13 above with reference to colorwiping. As shown it Fig:. 16A-C and 14 17 the position data is measured along the horizontal axis (Fig.
16A? and is transferred to the subtitle decoder with the subtitle 16 data during the appropriate frame (Fig. 16C~ c:arrespor.~ding to a 17 presentation time stamp (PTS(n~, for example; Fig. 16B).
18 The positioning operation will now be explained with 19 reference to Fig. 1,'7. The position data ~:~ a value representing the position of the subtitle frame along the horizontal axis and 21 is read out from the display buffer and latched by register 205 22 on each vertical sync pulse. Pixel counter 208 decrements the 23 position data on each horizontal sync p~z~.s~ arnd send a Boolean 24 flag to controller 35 (Fig. 2) to i.ndicat.e that the position of sony.l5\3582.app ~ 5 CA 02181864 1996-0~ ~8~
PATENT

1 the subtitle frame has not been reached. When the pixel counter 2 reaches zero, the position of the subtitle frame has been reached 3 and the Boolean flag is toggled to indicate this to the 4 controller. The controller, which has ~:~een delaying the reading operation of code buffer 22 (Fig. 2), then causes the code buffer 6 to read out the subtitle data to run length decoder 24 (Fig. 2).
7 The subtitle data is then decoded as described above and 8 displayed with the corresponding video image. In this manner, 9 the position of the subtitle frame is c~aanged with each frame;
thus providing dynamic movement of the subtitle frame 11 It will be appreciated that tree present invention is 12 applicable to other applications, ;such as television or video 13 graphics. It is, therefore, to be understood that, within the 14 scope of the appended claims, the invention may be pr<~cticed otherwise than as specifically describedAherein.
sony.l5\3582.app 4

Claims

1. A method of storing subtitles composed of bit streams of subtitle data on a record medium having sectors arranged by address, said subtitles being stored with encoded video data constituting a video picture composed of a series of video frames, comprising the steps of:
writing a first subtitle to be displayed with said video frames during a normal playback mode on the record medium at a first address;
writing said first address on said record medium in an area reserved for addresses of the record medium;
writing a second subtitle, different in textual content from said first subtitle, to be displayed during a trick playback mode, on the record medium at a second address;
and writing said second address on said record medium in said area reserved for addresses.

2. The method of storing subtitles of claim 1, wherein the step of writing a second subtitle further comprises writing the second subtitle as a page of bit streams of subtitle data beginning with a page header portion positioned at said second address on the record medium.

3. The method of storing subtitles of claim 2, wherein the step of writing said second subtitle further comprises encoding display time information within said page header portion indicating a display time for the second subtitle.

4. The method of storing subtitles of claim 1, further comprising the step of writing a table of contents on a contiguous area of sectors of said record medium preceding the encoded video data that stores said first and second addresses for assisting retrieval of said subtitles during playback.

5. The method of storing subtitles of claim 1, further comprising the step of writing, on said record medium, a stream map as one of said bit streams of subtitle data that stores a record of said first and second addresses with the encoded video data for assisting retrieval of said subtitles during playback.

6. The method of storing subtitles of claim 1, further comprising the step of writing said second address as one of said bit streams of subtitle data on said record medium.

7. An apparatus for writing subtitles composed of bit streams of subtitle data on a record medium having sectors arranged by address, said subtitles being recorded with encoded video data constituting a video picture composed of a series of video frames, comprising:
subtitle writing means for writing a first subtitle to be displayed with said video frames during a normal playback mode on the record medium at a first address, and for writing a second subtitle, different in textual content from said first subtitle, to be displayed during a trick playback mode on the record medium at a second address;
and multiplexing means for multiplexing the first subtitle, the second subtitle and the encoded video data;
wherein said subtitle writing means writes the multiplexed data on said record medium, said first and second subtitles being written at said first and second addresses, respectively, and writes the first and second addresses on said record medium in an area reserved for addresses on the record medium.

8. The apparatus of claim 7, wherein the subtitle writing means writes the second subtitle on said record medium as a page of bit streams of subtitle data beginning with a page header portion at said second address.

9. The apparatus of claim 8, wherein the subtitle writing means writes display tune information within said page header portion indicating a display time for the second subtitle during the normal playback mode.

10. The apparatus of claim 7, wherein said subtitle writing means further comprises means for writing a table of contents of the first and second addresses on a contiguous area of said record medium preceding the encoded video data for assisting retrieval of said subtitles during playback.

11. The apparatus of claim 7, wherein said subtitle writing means further comprises stream mapping means for writing, as one of said bit streams of subtitle data the first and second addresses on said record medium with the encoded video data, for assisting retrieval of said subtitles during playback.

12. The apparatus of claim 7, wherein the subtitle writing means writes the encoded second address in a bit stream of subtitle data on said record medium.

13. A computer-readable memory for directing a computer to play back a subtitle to be displayed during a trick playback mode comprising:
first pointing means for directing said computer during a normal playback mode to skip to an address pointing to a sector of said computer-readable memory where a first subtitle to be displayed with a respective video frame is stored and for causing said computer to read said first subtitle; and second pointing means for directing said computer during a trick playback mode to skip to an address pointing to a sector of said computer-readable memory where a second subtitle, being different in textual content from said first subtitle and to be displayed with said video frames, is stored and for causing said computer to read said second subtitle.

14. The computer-readable memory of claim 13, further comprising:
a driven pickup movable to different positions for reading said first and second subtitles from said computer readable memory in response to address signals produced by the first and second pointing means.

15. The computer-readable memory of claim 14, wherein said computer-readable memory is a digital video disc.

16. A method of reading subtitles composed of bit streams of subtitle data stored on a record medium having sectors arranged by address, said subtitles being encoded and recorded with video data constituting a video picture composed of a series of video frames to be displayed in normal and trick playback modes, comprising the steps of:

reading out a subtitle address stored on said record medium pointing to a subtitle to be displayed during said trick playback mode which is different in textual content than subtitles to be displayed in said normal playback mode;
reading out said subtitle stored on said record medium at said subtitle address read from said record medium; and decoding said subtitle for display with a respective video frame during said trick playback mode.

17. The method of claim 16, further comprising the steps of:
reading out display time data stored with the subtitles on said record medium indicating a duration in which subtitles are to be displayed in a video frame;
and varying said duration for different playback speeds of the trick playback mode of the video picture such that said subtitle is displayed for a time determined by a particular playback speed.

18. The method of claim 17, wherein the step of varying divides the display time data by twice the playback speed for the normal playback mode to speed up said duration as a function of the playback speed in trick playback mode.

19. The method of claim 16, further comprising the step of reading out the subtitle to be displayed in trick playback mode stored on said record medium as a page of bit streams of subtitle data beginning with a page header portion positioned at said subtitle address on the record medium.

20. The method of claim 16, further comprising the step of reading a table of contents stored on said record medium in a contiguous area of sectors preceding the encoded video data that stores addresses of the subtitles to be displayed in trick playback mode for assisting retrieval of said subtitles.

21. The method of claim 16, further comprising the step of reading, from said record medium, a stream map as one of said bit streams that stores the addresses of the subtitle to be displayed in the trick playback mode for assisting retrieval of said subtitles during decoding.

22. The method of claim 16, further comprising the steps of:
reading said subtitle address stored on said record medium from one of said bit streams of subtitle data; and decoding said subtitle address stored on said record medium as video encoded data for assisting retrieval of said subtitles.

23. An apparatus for recovering subtitles composed of bit streams of subtitle data from a record medium having sectors arranged by address, said subtitles being encoded and recorded with video data constituting a video picture composed of a series of video frames to be displayed in normal and trick playback modes, comprising:
means for reading, from said record medium, a subtitle address pointing to a subtitle to be displayed exclusively during said trick playback mode and for reading said subtitle at said subtitle address, wherein said subtitle to be displayed exclusively during said trick playback mode is different in textual content than subtitles to be displayed in said normal playback mode; and means for decoding said subtitle for display with said respective video frame during said trick playback mode.

24. The apparatus of claim 23, further comprising:
means for reading out display time data stored with the subtitles on said record medium indicating a duration in which the subtitle is to be displayed in a video frame;
and means for varying said duration for different playback speeds of the video picture such that said subtitle is displayed for a time determined by a particular playback speed.

25. The apparatus of claim 24, wherein said means for varying is a multiplier which multiplies said display time data by a factor for varying said duration.

26. The apparatus of claim 25, wherein said factor is the inverse of twice the particular playback speed for speeding up the duration as a function of the playback speed.

27. The apparatus of claim 23, further comprising means for reading the subtitle for display in the trick playback mode stored on said record medium as a page of bit streams of subtitle data beginning with a page header portion positioned at said subtitle address on the record medium.

28. The apparatus of claim 23, wherein said means for reading further reads a table of contents stored on said record medium in a contiguous area of sectors preceding the encoded video data which stores addresses of the subtitles to assist retrieval of said subtitles.

29. The apparatus of claim 23, wherein said means for reading further reads, from a stream map stored on the record medium, the addresses of the subtitles on the record medium for assisting retrieval of said subtitles.

30. The apparatus of claim 23, wherein said means for reading further reads said subtitle address as one of said bit streams of subtitle data from said record medium;
and wherein, the decoding means further decodes said subtitle address.

31. The apparatus of claim 30, further comprising a controller for driving a pick up to a sector of said subtitle address and for retrieving said subtitle from said record medium positioned at said subtitle address.