US8099277B2 - Speech-duration detector and computer program product therefor - Google Patents

Speech-duration detector and computer program product therefor Download PDF

Info

Publication number
US8099277B2
US8099277B2 US11/725,566 US72556607A US8099277B2 US 8099277 B2 US8099277 B2 US 8099277B2 US 72556607 A US72556607 A US 72556607A US 8099277 B2 US8099277 B2 US 8099277B2
Authority
US
United States
Prior art keywords
duration
speech
time length
characteristic
starting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US11/725,566
Other versions
US20080077400A1 (en
Inventor
Koichi Yamamoto
Akinori Kawamura
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Toshiba Digital Solutions Corp
Original Assignee
Toshiba Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp filed Critical Toshiba Corp
Assigned to KABUSHIKI KAISHA TOSHIBA reassignment KABUSHIKI KAISHA TOSHIBA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KAWAMURA, AKINORI, YAMAMOTO, KOICHI
Publication of US20080077400A1 publication Critical patent/US20080077400A1/en
Application granted granted Critical
Publication of US8099277B2 publication Critical patent/US8099277B2/en
Assigned to TOSHIBA DIGITAL SOLUTIONS CORPORATION reassignment TOSHIBA DIGITAL SOLUTIONS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KABUSHIKI KAISHA TOSHIBA
Assigned to KABUSHIKI KAISHA TOSHIBA, TOSHIBA DIGITAL SOLUTIONS CORPORATION reassignment KABUSHIKI KAISHA TOSHIBA CORRECTIVE ASSIGNMENT TO CORRECT THE ADD SECOND RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 48547 FRAME: 187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: KABUSHIKI KAISHA TOSHIBA
Assigned to TOSHIBA DIGITAL SOLUTIONS CORPORATION reassignment TOSHIBA DIGITAL SOLUTIONS CORPORATION CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S ADDRESS PREVIOUSLY RECORDED ON REEL 048547 FRAME 0187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Assignors: KABUSHIKI KAISHA TOSHIBA
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L25/87—Detection of discrete points within a voice signal

Definitions

  • the present invention relates to a speech-duration detector that detects a starting end and a trailing end of speech from an input acoustic signal, and to a computer program product for the detection.
  • a typical speech-duration detection method detects starting and trailing ends of a speech-duration based on rising/falling of an envelope of a short-time power (hereinafter, “power”) extracted for each frame of 20 to 40 milliseconds.
  • power a short-time power extracted for each frame of 20 to 40 milliseconds.
  • Such detection of starting and trailing ends of a speech-duration is carried out by using a finite state automaton (FSA) disclosed in Japanese Patent No. 3105465.
  • FSA finite state automaton
  • a countermeasure of reducing a trailing end detection time to be shorter than a time length from the correct trailing end to the extemporaneous noise can be considered for the problem.
  • a word including a double consonant e.g., “Sapporo” is detected as divided durations. That is, there is a problem that silence in a word cannot be discriminated from that after end of utterance.
  • a speech-duration detector includes a characteristic extracting unit that extracts a characteristic of an input acoustic signal; a starting-end detecting unit that detects a starting end of a first duration where the characteristic exceeds a threshold value as a starting end of a speech-duration, when the first duration continues for a first time length; a trailing-end-candidate detecting unit that detects a starting end of a second duration where the characteristic is lower than the threshold value as a candidate point for a trailing end of speech, when the second duration continues for a second time length after the starting end of the speech-duration is detected; and a trailing-end-candidate determining unit that determines the candidate point as a trailing end of the speech-duration, when the second duration where the characteristic exceeds the threshold value does not continue for the first time length while a third time length elapses from measurement at the candidate point.
  • a speech-duration detector includes a characteristic extracting unit that extracts a characteristic of an input acoustic signal; a starting-end-candidate detecting unit that detects a starting end of a third duration where the characteristic exceeds a threshold value as a candidate point for a starting point of speech, when the third duration continues for a fourth time length; a starting-end-candidate determining unit that determines the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point and a forth duration where the characteristic exceeds a threshold value continues for a fifth time length; and a trailing-end detecting unit that detects a starting end of a fifth duration where the characteristic is lower than the threshold value as a trailing end of the speech-duration, when the fifth duration continues for a sixth time length after the starting end of the speech-duration is determined.
  • a computer program product causes a computer to perform the method according to the present invention.
  • FIG. 1 is a block diagram showing a hardware configuration of a speech-duration detector according to a first embodiment of the present invention
  • FIG. 2 is a block diagram showing a functional configuration of the speech-duration detector
  • FIG. 3 is a state transition diagram of a configuration of a finite state automaton
  • FIG. 4 is a graph of an example of an observed power envelope and state transition of the finite state automaton
  • FIG. 5 is a block diagram of a functional configuration of a speech-duration detector according to a second embodiment of the present invention.
  • FIG. 6 is a state transition diagram of a configuration of a finite state automaton.
  • FIG. 7 is a graph of an example of an observed power envelope and state transition of the finite state automaton.
  • FIG. 1 is a block diagram of a hardware configuration of a speech-duration detector according to the first embodiment.
  • the speech-duration detector according to the embodiment generally uses a finite state automaton (FSA) to detect a starting and a trailing ends of a speech-duration.
  • FSA finite state automaton
  • the speech-duration detector 1 is, e.g., a personal computer, and includes a Central Processing Unit (CPU) 2 that is a primary unit of the computer and intensively controls each unit. To the CPU 2 are connected a Read Only Memory (ROM) 3 as a read only memory storing, e.g., BIOS therein and a Random Access Memory (RAM) 4 that rewritably stores various kinds of data through a bus 5 .
  • ROM Read Only Memory
  • BIOS e.g., BIOS therein
  • RAM Random Access Memory
  • HDD Hard Disk Drive
  • CD-ROM drive 8 that reads information in a Compact Disc (CD)-ROM 7 as a mechanism that reads computer software as a distributed program
  • communication controller 10 that controls communication between the speech-duration detector 1 and a network 9
  • an input device 11 e.g., a keyboard or a mouse that instructs various kinds of operations
  • a display unit 12 that displays various kinds of information, e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) via an I/O (not shown).
  • CTR Cathode Ray Tube
  • LCD Liquid Crystal Display
  • the RAM 4 Since the RAM 4 has properties of rewritably storing various kinds of data, it functions as a working area for the CPU 2 to serve as, e.g., a buffer.
  • the CD-ROM 7 shown in FIG. 1 realizes a storage medium in the present invention, and stores an Operating System (OS) or various kinds of programs.
  • the CPU 2 reads a program stored in the CD-ROM 7 by using the CD-ROM drive 8 , and installs it in the HDD 6 .
  • OS Operating System
  • a program may be downloaded from the network 9 , e.g., the Internet via the communication controller 10 to be installed in the HDD 6 .
  • a storage unit that stores the program in a server on a transmission side is also a storage medium in the present invention.
  • the program may operate in a predetermined Operating System (OS).
  • OS Operating System
  • the program may allow the OS to execute a part of after-mentioned various kinds of processing.
  • the program may be included as a part of a program file group constituting a predetermined application software or the OS.
  • the CPU 2 that controls operations of the entire system executes various kinds of processing based on the program loaded in the HDD 6 used as a main storage unit in the system.
  • FIG. 2 is a block diagram of a functional configuration of the speech-duration detector 1 .
  • the speech-duration detector 1 includes an A/D converter 21 that converts an input signal from an analog signal to a digital signal at a predetermined sampling frequency in compliance with a speech-duration detection program, a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames, a characteristic extractor 23 as a characteristic extracting unit that calculates a power from frames divided by the frame divider 22 , a finite state automaton (FSA) unit 24 that uses a power obtained by the characteristic extractor 23 to detect starting and trailing ends of speech, and a voice recognizer 25 that uses duration information from the FSA unit 24 to perform speech recognition processing.
  • A/D converter 21 that converts an input signal from an analog signal to a digital signal at a predetermined sampling frequency in compliance with a speech-duration detection program
  • a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames
  • the FSA unit 24 includes a starting-end detecting unit 241 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 exceeds a threshold value as a starting end of a speech-duration when the duration continues for a predetermined time, and a trailing-end detecting unit 242 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 is below a threshold value as a trailing end of a speech-duration when the duration continues for a predetermined time after the starting-end detecting unit 241 detects the starting end of the speech-duration.
  • the trailing-end detecting unit 242 includes a trailing-end-candidate detecting unit 243 that detects a candidate point for a speech trailing end, and a trailing-end-candidate determining unit 244 that determines a trailing-end candidate point detected by the trailing-end-candidate detecting unit 243 as a speech trailing end.
  • the A/D converter 21 converts an input signal required to detect a speech-duration into a digital signal from an analog signal.
  • the frame divider 22 divides the digital signal converted by the A/D converter 21 into frames each having a length of 20 to 30 milliseconds and an interval of approximately 10 to 20 milliseconds.
  • a hamming window may be used as a windowing function required to perform framing processing.
  • the characteristic extractor 23 extracts a power from an acoustic signal of each frame divided by the frame divider 22 .
  • the FSA unit 24 uses the power of each frame extracted by the characteristic extractor 23 to detect starting and trailing ends of speech, and carries out speech recognition processing with respect to a detected duration.
  • a finite state automaton (FSA) of the FSA unit 24 has four states, i.e., a noise state, a starting end detection state, a trailing-end-candidate detection state, and a trailing-end-candidate determination state.
  • the FSA of the FSA unit 24 uses a starting end detection time Ts as a first time length, a trailing-end-candidate detection time Te 1 as a second time length, and a trailing end determination time Te 2 as a third time length for detection of starting and trailing ends of speech.
  • Ts a first time length
  • a trailing-end-candidate detection time Te 1 as a second time length
  • a trailing end determination time Te 2 as a third time length for detection of starting and trailing ends of speech.
  • the noise state is determined as an initial state.
  • a power extracted from an input signal exceeds a threshold value 1 as a threshold value for starting end detection
  • a transition from the noise state to the starting end detection state is achieved.
  • the starting end detection state when a duration where a power is equal to or above the threshold value 1 continues for the starting end detection time T s , a starting end of the duration is determined as a starting end of speech, and the starting end detection state shifts to the trailing-end-candidate detection state.
  • the starting end detection time-T s is set to approximately 100 milliseconds to avoid an erroneous operation due to extemporaneous noise other than speech.
  • a position obtained by adding a preset offset may be determined as a final starting end position of speech. That is, when a starting end position detected by the automaton is a position that is T second behind a processing start position, a position obtained by adding a starting end offset F s , i.e., a position that is T+F s seconds behind may be determined as a final starting end position. When the starting end offset F s is negative, a position harked back to the past is determined as a final starting end of speech. When the starting end offset F s is positive, a position advanced to the future is determined as the same.
  • a threshold value 2 as a threshold value required to detect a trailing end is used to achieve a transition between the states of the FSA.
  • a magnitude of human voice is reduced toward a last half of utterance. Therefore, when a characteristic is a power, like the embodiment, a setting, e.g., the threshold value 1 >the threshold value 2 enables threshold value setting that is optimum for detection of a starting end and a trailing end.
  • the threshold value may be adaptively varied for each frame rather than setting a fixed value in advance.
  • trailing-end-candidate detection state when a duration where the power is lower than the threshold value 2 continues for the trailing-end-candidate detection time T e1 or more, a starting end of the duration is determined as a trailing-end-candidate point, and the trailing-end-candidate detection state shifts to the trailing-end-candidate determination state.
  • transmitting trailing end information to the voice recognizer 25 at a rear stage upon detection of the candidate point can improve responsiveness of the entire system.
  • the trailing-end-candidate determination state After transition between the states, when a duration where the power is equal to or above the threshold value 2 does not continue for the starting end detection time T s while the trailing end determination time T e2 elapses from measurement at the trailing-end-candidate point, the trailing-end-candidate point is determined as a trailing end of speech. In other cases, i.e., when the duration where the power is equal to or above the threshold value 2 continues for the starting end detection time T s , the trailing-end-candidate point detected in the trailing-end-candidate detection state is canceled, and the current state shifts to the trailing-end-candidate detection state.
  • a finally detected speech-duration length (a trailing end time instant—a starting end time instant) is shorter than a preset minimum speech-duration length T min , the detected duration is possibly extemporaneous noise, and the detected starting end and trailing end positions are thereby canceled to achieve a transition to the noise state. As a result, an accuracy can be improved.
  • the minimum speech-duration length T min is set to approximately 200 milliseconds.
  • two time continuation length parameters i.e., the candidate point detection time and the candidate point determination time are used for detection of a trailing end of speech.
  • the trailing-end-candidate detection state detection including a soundless duration in a word, e.g., a double consonant is intended.
  • the trailing-end-candidate determination state whether a candidate point detected in the trailing-end-candidate detection state corresponds to silence in a word, e.g., a double consonant or silence after end of utterance is judged.
  • the trailing-end-candidate detection time T e1 is set to approximately 120 milliseconds with a length that is equal to or longer than a soundless duration (double consonant) included in a word being determined as a rough standard
  • the trailing end determination time T e2 is set to approximately 400 milliseconds as a length representing an interval between utterances.
  • a position obtained by adding a trailing end offset Fe can be determined as a final speech trailing end position.
  • speech-duration detection is used as preprocessing of speech recognition, a positive offset value is usually provided in trailing end detection. As a result, missing an end of an uttered word can be avoided, thereby improving a speech recognition accuracy.
  • two time continuation length parameters i.e., the candidate point detection time and the candidate point determination time are used for detection of a trailing end of speech to provide two states, i.e., the candidate point detection state and the candidate point determination state for a trailing end of speech. Consequently, even if noise extemporaneously occurs after an appropriate trailing end (a correct trailing end) of a speech-duration as shown in FIG. 4 , a state transition shown in FIG. 4 enables detection of the correct speech trailing end. That is, according to the embodiment, silence in a word can be discriminated from silence after end of utterance.
  • Realizing high-performance speech-duration detection in this manner can improve speech recognition performance when the detection is used as, e.g., preprocessing of speech recognition.
  • a correct trailing end is detected, an unnecessary frame that can be a target of speech recognition processing can be eliminated. Therefore, not only a response speed with respect to speech can be increased but also an amount of calculation can be reduced.
  • a short-time power is used as a characteristic for each frame in the embodiment, but the present invention is not restricted thereto. Any other characteristic can be used.
  • a likelihood ratio of a voice model and a non-voice model is, used as a characteristic per predetermined time.
  • FIGS. 5 to 7 A second embodiment according to the present invention will now be explained with reference to FIGS. 5 to 7 . It is to be noted that same reference numerals denote parts equal to those in the first embodiment, thereby omitting an explanation thereof.
  • two states of, e.g., candidate point detection and candidate point determination are provided.
  • FIG. 5 is a block diagram of a functional configuration of a speech-duration detector 1 according to the second embodiment.
  • the speech-duration detector 1 includes an A/D converter 21 that converts an input signal into a digital signal from an analog signal at a predetermined sampling frequency in compliance with a speech-duration detection program, a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames, a characteristic extractor 23 that calculates a power from frames divided by the frame divider 22 , a finite state automaton (FSA) unit 30 that uses a power obtained by the characteristic extractor 23 to detect starting and trailing ends of speech, and a voice recognizer 25 that uses duration information from the FSA unit 30 to perform speech recognition processing.
  • A/D converter 21 that converts an input signal into a digital signal from an analog signal at a predetermined sampling frequency in compliance with a speech-duration detection program
  • a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames
  • the FSA unit 30 includes a starting-end detecting unit 301 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 exceeds a threshold value as a starting end of a speech-duration when the duration continues for a predetermined time, and a trailing-end detecting unit 302 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 is lower than the threshold value as a trailing end of a speech-duration when the duration continues for a predetermined time.
  • the starting-end detecting unit 301 includes a starting-end-candidate detecting unit 303 that detects a candidate point for a starting point of speech, and a starting-end-candidate determining unit 304 that determines a starting-end-candidate point detected by the starting-end-candidate detecting unit 303 as a starting end of speech.
  • the A/D converter 21 converts an input signal that is used to detect a speech-duration from an analog signal to a digital signal.
  • the frame divider 22 divides the digital signal converted by the A/D converter 21 into frames each having a length of 20 to 30 milliseconds and an interval of approximately 10 to 20 milliseconds.
  • a hamming window may be used as a windowing function that is required to perform framing processing.
  • the characteristic extractor 23 extracts a power from an acoustic signal of each frame divided by the frame divider 22 .
  • the FSA unit 30 uses the power of each frame extracted by the characteristic extractor 23 to detect a starting and a trailing ends of speech, and performs speech recognition processing with respect to the detected duration.
  • a finite state automaton (FSA) of the FSA unit 30 has four states, i.e., a noise state, a starting-end-candidate detection state, a starting-end-candidate determination state, and a trailing end detection state.
  • the finite state automaton (FSA) of the FSA unit 30 uses a starting-end-candidate detection time T s1 as a fourth time length, a starting end determination time T s2 as a fifth time length, and a trailing end detection time T e as a sixth time length in detection of a starting and a trailing ends of speech.
  • a transition between the states can be achieved based on comparison between an observed power and a preset threshold value.
  • the noise state is an initial state, and a transition to the starting-end-candidate detection state is achieved when a power extracted from an input signal exceeds a threshold value for detection of a starting and a trailing ends.
  • a threshold value for the power is set as a fixed value in advance, but also the threshold value may be adaptively varied for each frame.
  • the starting-end-candidate detection state when a duration where the power is equal to or above the threshold value continues for the starting-end-candidate detection time T s1 , a starting end of the duration is detected as a starting-end-candidate point of speech, and the current state shifts to the starting-end-candidate determination state.
  • the starting-end-candidate detection state when the power is lower than the threshold value, the current state shifts to the noise state as the initial state.
  • information of the detected starting-end-candidate point is transmitted to the voice recognizer 25 on a rear stage to start speech recognition processing from a frame where the starting-end-candidate point is detected.
  • the starting-end-candidate determination state when counting starts from the starting-end-candidate point and a duration where the power exceeds the threshold value, continues for the starting-end-candidate determination time T s2 , the starting-end-candidate point is determined as a starting end of speech, and the current state shifts to the trailing end detection state.
  • the starting-end-candidate determinations state when the power is lower than the threshold value, the detected starting-end-candidate point is canceled, speech recognition processing on the rear stage is stopped, and initialization is carried out, thereby achieving a transition to the starting-end-candidate detection state.
  • the starting-end-candidate detection time T s1 is set to approximately 20 milliseconds
  • the starting-end-candidate determination time T s2 is set to approximately 100 milliseconds.
  • a configuration of detecting and determining a candidate point is adopted for detection of a starting end, and speech recognition processing on the rear stage is started when the candidate point is detected.
  • a response time of (T s2 ⁇ T s1 ) milliseconds can be gained as compared with a conventional technology.
  • speech-duration detection is often used as preprocessing of, e.g., speech recognition. If detected speech-duration information can be rapidly transmitted to the voice recognizer 25 on the rear stage, responsiveness of entire speech recognition can be improved.
  • T s is simply reduced in the conventional technology, erroneous detection of a starting end is increased due to an influence of, e.g., extemporaneous noise.
  • the voice recognizer 25 performs characteristic amount extraction and decoder processing for speech recognition with respect to a frame from the starting end to the trailing end detected by the FSA unit 30 .
  • a finally detected speech-duration length (a trailing end time instance—a staring end time instance) is shorter than a preset minimum speech-duration length T min , the detected duration possibly corresponds to extemporaneous noise, and the detected starting and trailing end positions are thereby canceled to achieve a transition to the noise state. Consequently, an accuracy can be improved.
  • the minimum speech-duration length T min is set to approximately 200 milliseconds.
  • a candidate point alone is detected in regard to a starting point in the embodiment, but a candidate point can be likewise detected with respect to a trailing end by using such a technique as explained in conjunction with the first embodiment.

Abstract

A speech-duration detector includes a starting-end detecting unit that detects a starting end of a first duration where the characteristic exceeds a threshold value as a starting end of a speech-duration, when the first duration continues for a first time length; a trailing-end-candidate detecting unit that detects a starting end of a second duration where the characteristic is lower than the threshold value as a candidate point for a trailing end of speech, when the second duration continues for a second time length; and a trailing-end-candidate determining unit that determines the candidate point as a trailing end of the speech-duration, when the second duration where the characteristic exceeds the threshold value does not continue for the first time length while a third time length elapses from measurement at the candidate point.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2006-263113, filed on Sep. 27, 2006; the entire contents of which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a speech-duration detector that detects a starting end and a trailing end of speech from an input acoustic signal, and to a computer program product for the detection.
2. Description of the Related Art
A typical speech-duration detection method (a speech-duration detector) detects starting and trailing ends of a speech-duration based on rising/falling of an envelope of a short-time power (hereinafter, “power”) extracted for each frame of 20 to 40 milliseconds. Such detection of starting and trailing ends of a speech-duration is carried out by using a finite state automaton (FSA) disclosed in Japanese Patent No. 3105465.
However, according to the finite state automaton disclosed in Japanese Patent No. 3105465, a single time control parameter is used to detect each of starting and trailing ends. When noise extemporaneously occurs after an appropriate trailing end (a correct trailing end) of a speech-duration, a trailing end to be detected is disadvantageously detected in regard of the correct trailing end due to an influence of a power of the extemporaneous noise.
It is to be noted that a countermeasure of reducing a trailing end detection time to be shorter than a time length from the correct trailing end to the extemporaneous noise can be considered for the problem. When the trailing end detection time is simply reduced, however, a word including a double consonant, e.g., “Sapporo” is detected as divided durations. That is, there is a problem that silence in a word cannot be discriminated from that after end of utterance.
SUMMARY OF THE INVENTION
According to one aspect of the present invention, a speech-duration detector includes a characteristic extracting unit that extracts a characteristic of an input acoustic signal; a starting-end detecting unit that detects a starting end of a first duration where the characteristic exceeds a threshold value as a starting end of a speech-duration, when the first duration continues for a first time length; a trailing-end-candidate detecting unit that detects a starting end of a second duration where the characteristic is lower than the threshold value as a candidate point for a trailing end of speech, when the second duration continues for a second time length after the starting end of the speech-duration is detected; and a trailing-end-candidate determining unit that determines the candidate point as a trailing end of the speech-duration, when the second duration where the characteristic exceeds the threshold value does not continue for the first time length while a third time length elapses from measurement at the candidate point.
According to another aspect of the present invention, a speech-duration detector includes a characteristic extracting unit that extracts a characteristic of an input acoustic signal; a starting-end-candidate detecting unit that detects a starting end of a third duration where the characteristic exceeds a threshold value as a candidate point for a starting point of speech, when the third duration continues for a fourth time length; a starting-end-candidate determining unit that determines the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point and a forth duration where the characteristic exceeds a threshold value continues for a fifth time length; and a trailing-end detecting unit that detects a starting end of a fifth duration where the characteristic is lower than the threshold value as a trailing end of the speech-duration, when the fifth duration continues for a sixth time length after the starting end of the speech-duration is determined.
A computer program product according to still another aspect of the present invention causes a computer to perform the method according to the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram showing a hardware configuration of a speech-duration detector according to a first embodiment of the present invention;
FIG. 2 is a block diagram showing a functional configuration of the speech-duration detector;
FIG. 3 is a state transition diagram of a configuration of a finite state automaton;
FIG. 4 is a graph of an example of an observed power envelope and state transition of the finite state automaton;
FIG. 5 is a block diagram of a functional configuration of a speech-duration detector according to a second embodiment of the present invention;
FIG. 6 is a state transition diagram of a configuration of a finite state automaton; and
FIG. 7 is a graph of an example of an observed power envelope and state transition of the finite state automaton.
 DETAILED DESCRIPTION OF THE INVENTION
A first embodiment according to the present invention will now be explained with reference to FIGS. 1 to 4. FIG. 1 is a block diagram of a hardware configuration of a speech-duration detector according to the first embodiment. The speech-duration detector according to the embodiment generally uses a finite state automaton (FSA) to detect a starting and a trailing ends of a speech-duration.
As shown in FIG. 1, the speech-duration detector 1 is, e.g., a personal computer, and includes a Central Processing Unit (CPU) 2 that is a primary unit of the computer and intensively controls each unit. To the CPU 2 are connected a Read Only Memory (ROM) 3 as a read only memory storing, e.g., BIOS therein and a Random Access Memory (RAM) 4 that rewritably stores various kinds of data through a bus 5.
To the bus 5 are connected a Hard Disk Drive (HDD) 6 that stores various kinds of programs, a CD-ROM drive 8 that reads information in a Compact Disc (CD)-ROM 7 as a mechanism that reads computer software as a distributed program, a communication controller 10 that controls communication between the speech-duration detector 1 and a network 9, an input device 11, e.g., a keyboard or a mouse that instructs various kinds of operations, a display unit 12 that displays various kinds of information, e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) via an I/O (not shown).
Since the RAM 4 has properties of rewritably storing various kinds of data, it functions as a working area for the CPU 2 to serve as, e.g., a buffer.
The CD-ROM 7 shown in FIG. 1 realizes a storage medium in the present invention, and stores an Operating System (OS) or various kinds of programs. The CPU 2 reads a program stored in the CD-ROM 7 by using the CD-ROM drive 8, and installs it in the HDD 6.
It is to be noted that, as a storage medium, various kinds of optical disks such as a DVD, various kinds of magneto optical disks, various kinds of magnetic disks such as a flexible disk, and medias adopting various kinds of modes such as a semiconductor memory can be used as well as the CD-ROM 7. A program may be downloaded from the network 9, e.g., the Internet via the communication controller 10 to be installed in the HDD 6. In this case, a storage unit that stores the program in a server on a transmission side is also a storage medium in the present invention. It is to be noted that the program may operate in a predetermined Operating System (OS). In this case, the program may allow the OS to execute a part of after-mentioned various kinds of processing. Alternatively, the program may be included as a part of a program file group constituting a predetermined application software or the OS.
The CPU 2 that controls operations of the entire system executes various kinds of processing based on the program loaded in the HDD 6 used as a main storage unit in the system.
Of functions executed by the CPU 2 based on various kinds of programs installed in the HDD 6 of the speech-duration detector 1, characteristic functions of the speech-duration detector 1 according to the embodiment will now be explained.
FIG. 2 is a block diagram of a functional configuration of the speech-duration detector 1. As shown in FIG. 2, the speech-duration detector 1 includes an A/D converter 21 that converts an input signal from an analog signal to a digital signal at a predetermined sampling frequency in compliance with a speech-duration detection program, a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames, a characteristic extractor 23 as a characteristic extracting unit that calculates a power from frames divided by the frame divider 22, a finite state automaton (FSA) unit 24 that uses a power obtained by the characteristic extractor 23 to detect starting and trailing ends of speech, and a voice recognizer 25 that uses duration information from the FSA unit 24 to perform speech recognition processing.
The FSA unit 24 includes a starting-end detecting unit 241 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 exceeds a threshold value as a starting end of a speech-duration when the duration continues for a predetermined time, and a trailing-end detecting unit 242 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 is below a threshold value as a trailing end of a speech-duration when the duration continues for a predetermined time after the starting-end detecting unit 241 detects the starting end of the speech-duration. The trailing-end detecting unit 242 includes a trailing-end-candidate detecting unit 243 that detects a candidate point for a speech trailing end, and a trailing-end-candidate determining unit 244 that determines a trailing-end candidate point detected by the trailing-end-candidate detecting unit 243 as a speech trailing end.
A procedure of the processing will now be explained hereinafter. First, the A/D converter 21 converts an input signal required to detect a speech-duration into a digital signal from an analog signal. Then, the frame divider 22 divides the digital signal converted by the A/D converter 21 into frames each having a length of 20 to 30 milliseconds and an interval of approximately 10 to 20 milliseconds. At this time, a hamming window may be used as a windowing function required to perform framing processing. Then, the characteristic extractor 23 extracts a power from an acoustic signal of each frame divided by the frame divider 22. Thereafter, the FSA unit 24 uses the power of each frame extracted by the characteristic extractor 23 to detect starting and trailing ends of speech, and carries out speech recognition processing with respect to a detected duration.
The FSA unit 24 will now be explained in detail. As shown in FIG. 3, a finite state automaton (FSA) of the FSA unit 24 has four states, i.e., a noise state, a starting end detection state, a trailing-end-candidate detection state, and a trailing-end-candidate determination state. The FSA of the FSA unit 24 uses a starting end detection time Ts as a first time length, a trailing-end-candidate detection time Te1 as a second time length, and a trailing end determination time Te2 as a third time length for detection of starting and trailing ends of speech. Such an FSA in the FSA unit 24 realizes a transition between the states based on comparison between an observed power and a preset threshold value.
In the FSA shown in FIG. 3, the noise state is determined as an initial state. When a power extracted from an input signal exceeds a threshold value 1 as a threshold value for starting end detection, a transition from the noise state to the starting end detection state is achieved. In the starting end detection state, when a duration where a power is equal to or above the threshold value 1 continues for the starting end detection time Ts, a starting end of the duration is determined as a starting end of speech, and the starting end detection state shifts to the trailing-end-candidate detection state. Here, the starting end detection time-Ts is set to approximately 100 milliseconds to avoid an erroneous operation due to extemporaneous noise other than speech. At this time, a position obtained by adding a preset offset may be determined as a final starting end position of speech. That is, when a starting end position detected by the automaton is a position that is T second behind a processing start position, a position obtained by adding a starting end offset Fs, i.e., a position that is T+Fs seconds behind may be determined as a final starting end position. When the starting end offset Fs is negative, a position harked back to the past is determined as a final starting end of speech. When the starting end offset Fs is positive, a position advanced to the future is determined as the same. When speech-duration detection is used as preprocessing of speech recognition, missing an anlaut of speech at a speech-duration detection stage does not lead to restoration of information, thereby deteriorating speech recognition performance. Thus, in detection of a starting end, giving a negative offset value enables extensive detection of a starting end of speech in a direction of the past. As a result, missing a starting end of speech can be avoided, thereby improving a speech recognition accuracy. In the starting end detection state, when the power is lower than the threshold value 1, the state shifts to the noise state as the initial state. This is a series of processing of detecting a starting end of speech.
Detection of a trailing end of speech will now be explained. In the trailing-end-candidate detection state, a threshold value 2 as a threshold value required to detect a trailing end is used to achieve a transition between the states of the FSA. In general, a magnitude of human voice is reduced toward a last half of utterance. Therefore, when a characteristic is a power, like the embodiment, a setting, e.g., the threshold value 1>the threshold value 2 enables threshold value setting that is optimum for detection of a starting end and a trailing end. As another threshold value setting method, the threshold value may be adaptively varied for each frame rather than setting a fixed value in advance. In the trailing-end-candidate detection state, when a duration where the power is lower than the threshold value 2 continues for the trailing-end-candidate detection time Te1 or more, a starting end of the duration is determined as a trailing-end-candidate point, and the trailing-end-candidate detection state shifts to the trailing-end-candidate determination state. In this case, transmitting trailing end information to the voice recognizer 25 at a rear stage upon detection of the candidate point can improve responsiveness of the entire system.
In the trailing-end-candidate determination state, after transition between the states, when a duration where the power is equal to or above the threshold value 2 does not continue for the starting end detection time Ts while the trailing end determination time Te2 elapses from measurement at the trailing-end-candidate point, the trailing-end-candidate point is determined as a trailing end of speech. In other cases, i.e., when the duration where the power is equal to or above the threshold value 2 continues for the starting end detection time Ts, the trailing-end-candidate point detected in the trailing-end-candidate detection state is canceled, and the current state shifts to the trailing-end-candidate detection state. When a finally detected speech-duration length (a trailing end time instant—a starting end time instant) is shorter than a preset minimum speech-duration length Tmin, the detected duration is possibly extemporaneous noise, and the detected starting end and trailing end positions are thereby canceled to achieve a transition to the noise state. As a result, an accuracy can be improved. As a rough standard of a minimum unit for utterance, the minimum speech-duration length Tmin is set to approximately 200 milliseconds.
As explained above, according to the embodiment, two time continuation length parameters, i.e., the candidate point detection time and the candidate point determination time are used for detection of a trailing end of speech. Here, in the trailing-end-candidate detection state, detection including a soundless duration in a word, e.g., a double consonant is intended. In the trailing-end-candidate determination state, whether a candidate point detected in the trailing-end-candidate detection state corresponds to silence in a word, e.g., a double consonant or silence after end of utterance is judged.
It is to be noted that the trailing-end-candidate detection time Te1 is set to approximately 120 milliseconds with a length that is equal to or longer than a soundless duration (double consonant) included in a word being determined as a rough standard, and the trailing end determination time Te2 is set to approximately 400 milliseconds as a length representing an interval between utterances.
In detection of a trailing end, like detection of a starting end, a position obtained by adding a trailing end offset Fe can be determined as a final speech trailing end position. When speech-duration detection is used as preprocessing of speech recognition, a positive offset value is usually provided in trailing end detection. As a result, missing an end of an uttered word can be avoided, thereby improving a speech recognition accuracy.
As explained above, according to the embodiment, two time continuation length parameters, i.e., the candidate point detection time and the candidate point determination time are used for detection of a trailing end of speech to provide two states, i.e., the candidate point detection state and the candidate point determination state for a trailing end of speech. Consequently, even if noise extemporaneously occurs after an appropriate trailing end (a correct trailing end) of a speech-duration as shown in FIG. 4, a state transition shown in FIG. 4 enables detection of the correct speech trailing end. That is, according to the embodiment, silence in a word can be discriminated from silence after end of utterance.
Realizing high-performance speech-duration detection in this manner can improve speech recognition performance when the detection is used as, e.g., preprocessing of speech recognition. When a correct trailing end is detected, an unnecessary frame that can be a target of speech recognition processing can be eliminated. Therefore, not only a response speed with respect to speech can be increased but also an amount of calculation can be reduced.
It is to be noted that a short-time power is used as a characteristic for each frame in the embodiment, but the present invention is not restricted thereto. Any other characteristic can be used. For example, in Patent Document 1, a likelihood ratio of a voice model and a non-voice model is, used as a characteristic per predetermined time.
A second embodiment according to the present invention will now be explained with reference to FIGS. 5 to 7. It is to be noted that same reference numerals denote parts equal to those in the first embodiment, thereby omitting an explanation thereof.
According to the embodiment, in detection of a starting end of speech, two states of, e.g., candidate point detection and candidate point determination are provided.
FIG. 5 is a block diagram of a functional configuration of a speech-duration detector 1 according to the second embodiment. As shown in FIG. 5, the speech-duration detector 1 according to the embodiment includes an A/D converter 21 that converts an input signal into a digital signal from an analog signal at a predetermined sampling frequency in compliance with a speech-duration detection program, a frame divider 22 that divides a digital signal output from the A/D converter 21 into frames, a characteristic extractor 23 that calculates a power from frames divided by the frame divider 22, a finite state automaton (FSA) unit 30 that uses a power obtained by the characteristic extractor 23 to detect starting and trailing ends of speech, and a voice recognizer 25 that uses duration information from the FSA unit 30 to perform speech recognition processing.
The FSA unit 30 includes a starting-end detecting unit 301 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 exceeds a threshold value as a starting end of a speech-duration when the duration continues for a predetermined time, and a trailing-end detecting unit 302 that detects a starting end of a duration where a characteristic extracted by the characteristic extractor 23 is lower than the threshold value as a trailing end of a speech-duration when the duration continues for a predetermined time. The starting-end detecting unit 301 includes a starting-end-candidate detecting unit 303 that detects a candidate point for a starting point of speech, and a starting-end-candidate determining unit 304 that determines a starting-end-candidate point detected by the starting-end-candidate detecting unit 303 as a starting end of speech.
A procedure of processing will now be explained hereinafter. First, the A/D converter 21 converts an input signal that is used to detect a speech-duration from an analog signal to a digital signal. Then, the frame divider 22 divides the digital signal converted by the A/D converter 21 into frames each having a length of 20 to 30 milliseconds and an interval of approximately 10 to 20 milliseconds. At this time, a hamming window may be used as a windowing function that is required to perform framing processing. Subsequently, the characteristic extractor 23 extracts a power from an acoustic signal of each frame divided by the frame divider 22. Thereafter, the FSA unit 30 uses the power of each frame extracted by the characteristic extractor 23 to detect a starting and a trailing ends of speech, and performs speech recognition processing with respect to the detected duration.
The FSA unit 30 will now be explained in detail. As shown in FIG. 6, a finite state automaton (FSA) of the FSA unit 30 has four states, i.e., a noise state, a starting-end-candidate detection state, a starting-end-candidate determination state, and a trailing end detection state. The finite state automaton (FSA) of the FSA unit 30 uses a starting-end-candidate detection time Ts1 as a fourth time length, a starting end determination time Ts2 as a fifth time length, and a trailing end detection time Te as a sixth time length in detection of a starting and a trailing ends of speech. In such an FSA of the FSA unit 30, a transition between the states can be achieved based on comparison between an observed power and a preset threshold value.
In the FSA shown in FIG. 6, the noise state is an initial state, and a transition to the starting-end-candidate detection state is achieved when a power extracted from an input signal exceeds a threshold value for detection of a starting and a trailing ends. Here, not only the threshold value for the power is set as a fixed value in advance, but also the threshold value may be adaptively varied for each frame.
In the starting-end-candidate detection state, when a duration where the power is equal to or above the threshold value continues for the starting-end-candidate detection time Ts1, a starting end of the duration is detected as a starting-end-candidate point of speech, and the current state shifts to the starting-end-candidate determination state. On the other hand, in the starting-end-candidate detection state, when the power is lower than the threshold value, the current state shifts to the noise state as the initial state. At this time, information of the detected starting-end-candidate point is transmitted to the voice recognizer 25 on a rear stage to start speech recognition processing from a frame where the starting-end-candidate point is detected.
In the starting-end-candidate determination state, when counting starts from the starting-end-candidate point and a duration where the power exceeds the threshold value, continues for the starting-end-candidate determination time Ts2, the starting-end-candidate point is determined as a starting end of speech, and the current state shifts to the trailing end detection state. On the other hand, in the starting-end-candidate determinations state, when the power is lower than the threshold value, the detected starting-end-candidate point is canceled, speech recognition processing on the rear stage is stopped, and initialization is carried out, thereby achieving a transition to the starting-end-candidate detection state. Here, the starting-end-candidate detection time Ts1 is set to approximately 20 milliseconds, and the starting-end-candidate determination time Ts2 is set to approximately 100 milliseconds.
As explained above, a configuration of detecting and determining a candidate point is adopted for detection of a starting end, and speech recognition processing on the rear stage is started when the candidate point is detected. As a result, as shown in FIG. 7, a response time of (Ts2−Ts1) milliseconds can be gained as compared with a conventional technology. In general, speech-duration detection is often used as preprocessing of, e.g., speech recognition. If detected speech-duration information can be rapidly transmitted to the voice recognizer 25 on the rear stage, responsiveness of entire speech recognition can be improved. It is to be noted that, when the starting end detection time Ts is simply reduced in the conventional technology, erroneous detection of a starting end is increased due to an influence of, e.g., extemporaneous noise.
On the other hand, in the trailing end detection state, when a duration where the power is lower than the threshold value continues for the trailing end detection time Te, a starting end of the duration is detected as a trailing end of speech, and information about the detection is transmitted to the voice recognizer 25 on the rear stage. The voice recognizer 25 performs characteristic amount extraction and decoder processing for speech recognition with respect to a frame from the starting end to the trailing end detected by the FSA unit 30.
When a finally detected speech-duration length (a trailing end time instance—a staring end time instance) is shorter than a preset minimum speech-duration length Tmin, the detected duration possibly corresponds to extemporaneous noise, and the detected starting and trailing end positions are thereby canceled to achieve a transition to the noise state. Consequently, an accuracy can be improved. As a rough standard of a minimum unit for utterance, the minimum speech-duration length Tmin is set to approximately 200 milliseconds.
It is to be noted that a candidate point alone is detected in regard to a starting point in the embodiment, but a candidate point can be likewise detected with respect to a trailing end by using such a technique as explained in conjunction with the first embodiment.
Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

Claims (15)

1. A speech-duration detector comprising:
a characteristic extracting unit that extracts a characteristic of an input acoustic signal;
a starting-end detecting unit that detects a starting end of a first duration where the characteristic is equal to or greater than a threshold value as a starting end of a speech-duration, when the first duration continues for a first time length;
a trailing-end-candidate detecting unit that detects a starting end of a second duration where the characteristic is lower than the threshold value as a candidate point for a trailing end of speech, when the second duration continues for a second time length; and
a trailing-end-candidate determining unit that (i) determines the candidate point as a trailing end of the speech-duration, when a third duration where the characteristic is equal to or greater than the threshold value does not continue for a preset time length while a third time length elapses from measurement at the candidate point, and (ii) cancels the candidate point as a trailing end of the speech-duration, when the third duration where the characteristic is equal to or greater than the threshold value continues for more than the preset time length while the third time length elapses from measurement at the candidate point.
2. The speech-duration detector according to claim 1, wherein the second time length and the third time length are different from each other.
3. The speech-duration detector according to claim 1, wherein the trailing-end-candidate determining unit determines a position obtained by adding an offset to the determined trailing end of the speech-duration as a final trailing end of the speech-duration.
4. The speech-duration detector according to claim 1, wherein a position of the detected starting end and a position of the detected trailing end of the speech-duration are rejected, when a time length of the speech-duration from the detected starting end to the detected trailing end is smaller than a preset minimum speech-duration length.
5. A speech-duration detector comprising:
a characteristic extracting unit that extracts a characteristic of an input acoustic signal;
a starting-end-candidate detecting unit that (i) detects a starting end of a first duration where the characteristic is equal to or greater than a threshold value as a candidate point for a starting point of speech, when the first duration continues for a first time length;
a starting-end-candidate determining unit that (i) determines the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point and a second duration where the characteristic is equal to or greater than the threshold value continues for a second time length, and (ii) cancels the candidate point as a starting end of speech-duration, when the second duration does not continue for the second time length; and
a trailing-end detecting unit that detects a starting end of a third duration where the characteristic is lower than the threshold value as a trailing end of the speech-duration, when the third duration continues for a third time length.
6. The speech-duration detector according to claim 5, wherein the first time length and the second time length are different from each other.
7. The speech-duration detector according to claim 5, wherein the starting-end-candidate determining unit determines a position obtained by adding an offset to the determined starting end of the speech-duration as a final starting end of the speech-duration.
8. The speech-duration detector according to claim 5, wherein a position of the detected starting end and a position of the detected trailing end of the speech-duration are rejected, when a time length of the speech-duration from the detected starting end to the detected trailing end is shorter than a preset minimum speech-duration length.
9. A computer program product comprising a non-transitory computer readable medium including program instructions for detecting speech-duration, wherein the instructions, when executed by a computer, cause the computer to perform steps comprising:
extracting a characteristic of an input acoustic signal;
detecting a starting end of a first duration where the characteristic is equal to or greater than a threshold value as a starting end of a speech-duration, when the first duration continues for a first time length;
detecting a starting end of a second duration where the characteristic is lower than the threshold value as a candidate point, when the second duration continues for a second time length;
determining the candidate point as a trailing end of the speech-duration, when a third duration where the characteristic is equal to or greater than the threshold value does not continue for a preset time length while a third time length elapses from measurement at the candidate point; and
canceling the candidate point as a trailing end of the speech-duration, when the third duration where the characteristic is equal to or greater than the threshold value continues for more than the preset time length while the third time length elapses from measurement at the candidate point.
10. A computer program product comprising a non-transitory computer readable medium including program instructions for detecting speech-duration, wherein the instructions, when executed by a computer, cause the computer to perform steps comprising:
extracting a characteristic of an input acoustic signal;
detecting a starting end of a first duration where the characteristic is equal to or greater than a threshold value as a candidate point, when the first duration continues for a first time length;
determining the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point for the starting end of speech and a second duration where the characteristic is equal to or greater than the threshold value continues for a second time length;
canceling the candidate point as a starting end of a speech-duration, when the second duration does not continue for the second time length; and
detecting a starting end of a third duration where the characteristic is lower than the threshold value as a trailing end of the speech-duration, when the third duration continues for a third time length.
11. A speech-duration detector comprising:
a characteristic extracting unit that extracts a characteristic of an input acoustic signal;
a starting-end detecting unit that detects a starting end of a first duration where the characteristic is equal to or greater than a first threshold value as a starting end for a speech-duration, when the first duration continues for a first time length;
a trailing-end-candidate detecting unit that detects a starting end of a second duration where the characteristic is lower than a second threshold value as a candidate point for a trailing end of speech, when the second duration continues for a second time length;
a trailing-end-candidate determining unit that (i) determines the candidate point as a trailing end of the speech-duration, when a third duration where the characteristic is equal to or greater than the second threshold value does not continue for a preset time length while a third time length elapses from measurement at the candidate point, and (ii) cancels the candidate point as a trailing end of the speech-duration, when the third duration where the characteristic is equal to or greater than the second threshold value continues for more than the preset time length while the third time length elapses from measurement at the candidate point.
12. The speech-duration detector according to claim 11, wherein the second threshold is smaller than the first threshold.
13. A speech-duration detector comprising:
a characteristic extracting unit that extracts a characteristic of an input acoustic signal;
a starting-end candidate detecting unit that (i) detects a starting end of a first duration where the characteristic is equal to greater than a first threshold value as a candidate point for a starting point of speech, when the first duration continues for a first time length;
a starting-end-candidate determining unit that (i) determines the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point and a second duration where the characteristic is equal to or greater than the first threshold value continues for a second time length, and (ii) cancels the candidate point as a starting end of speech-duration, when the second duration does not continue for the second time length; and
a trailing-end detecting unit that detects a starting end of a third duration where the characteristic is lower than a second threshold value as a trailing end of the speech-duration, when the third duration continues for a third time length.
14. A computer program product comprising a non-transitory computer readable medium including program instructions for detecting speech-duration, wherein the instructions, when executed by a computer, cause the computer to perform steps comprising:
extracting a characteristic of an input acoustic signal;
detecting a starting end of a first duration where the characteristic is equal to or greater than a first threshold value as a starting end of a speech-duration, when the first duration continues for a first time length;
detecting a starting end of a second duration where the characteristic is lower than a second threshold value as a candidate point, when the second duration continues for a second time length;
determining the candidate point as a trailing end of the speech-duration, when a third duration where the characteristic is equal to or greater than the second threshold value does not continue for a preset time length while a third time length elapses from measurement at the candidate point; and
cancelling the candidate point as a trailing end of the speech-duration, when the third duration where the characteristic is equal to or greater than the second threshold value continues for more than the preset time length while the third time length elapses from measurement at the candidate point.
15. A computer program product comprising a non-transitory computer readable medium including program instructions for detecting speech-duration, wherein the instructions, when executed by a computer, cause the computer to perform steps comprising:
extracting a characteristic of an input acoustic signal;
detecting a starting end of a first duration where the characteristic is equal to or greater than a first threshold value as a candidate point, where the first duration continues for a first time length;
determining the candidate point as a starting end of a speech-duration, when measurement starts from the candidate point for the starting end of speech and a second duration where the characteristic is equal to or greater than the first threshold value continues for a second time length;
canceling the candidate point as a starting end of speech-duration, when the second duration does not continue for the second time length; and
detecting a starting end of a third duration where the characteristic is lower than a second threshold value as a trailing end of the speech-duration, when the third duration continues for a third time length.
US11/725,566 2006-09-27 2007-03-20 Speech-duration detector and computer program product therefor Active 2030-01-16 US8099277B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2006-263113 2006-09-27
JP2006263113A JP4282704B2 (en) 2006-09-27 2006-09-27 Voice section detection apparatus and program

Publications (2)

Publication Number Publication Date
US20080077400A1 US20080077400A1 (en) 2008-03-27
US8099277B2 true US8099277B2 (en) 2012-01-17

Family

ID=39226157

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/725,566 Active 2030-01-16 US8099277B2 (en) 2006-09-27 2007-03-20 Speech-duration detector and computer program product therefor

Country Status (3)

Country Link
US (1) US8099277B2 (en)
JP (1) JP4282704B2 (en)
CN (1) CN101154378A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090254341A1 (en) * 2008-04-03 2009-10-08 Kabushiki Kaisha Toshiba Apparatus, method, and computer program product for judging speech/non-speech
US20130282367A1 (en) * 2010-12-24 2013-10-24 Huawei Technologies Co., Ltd. Method and apparatus for performing voice activity detection
US20140379345A1 (en) * 2013-06-20 2014-12-25 Electronic And Telecommunications Research Institute Method and apparatus for detecting speech endpoint using weighted finite state transducer
US9818407B1 (en) * 2013-02-07 2017-11-14 Amazon Technologies, Inc. Distributed endpointing for speech recognition
US20180144740A1 (en) * 2016-11-22 2018-05-24 Knowles Electronics, Llc Methods and systems for locating the end of the keyword in voice sensing
US10546576B2 (en) * 2014-04-23 2020-01-28 Google Llc Speech endpointing based on word comparisons
US10832005B1 (en) 2013-11-21 2020-11-10 Soundhound, Inc. Parsing to determine interruptible state in an utterance by detecting pause duration and complete sentences

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4667082B2 (en) * 2005-03-09 2011-04-06 キヤノン株式会社 Speech recognition method
US20090198490A1 (en) * 2008-02-06 2009-08-06 International Business Machines Corporation Response time when using a dual factor end of utterance determination technique
US20110160887A1 (en) * 2008-08-20 2011-06-30 Pioneer Corporation Information generating apparatus, information generating method and information generating program
JP5834449B2 (en) * 2010-04-22 2015-12-24 富士通株式会社 Utterance state detection device, utterance state detection program, and utterance state detection method
JP2012150237A (en) 2011-01-18 2012-08-09 Sony Corp Sound signal processing apparatus, sound signal processing method, and program
WO2013005248A1 (en) * 2011-07-05 2013-01-10 三菱電機株式会社 Voice recognition device and navigation device
JP2015102702A (en) * 2013-11-26 2015-06-04 日本電信電話株式会社 Utterance section extraction device, method of the same and program
JP6459330B2 (en) * 2014-09-17 2019-01-30 株式会社デンソー Speech recognition apparatus, speech recognition method, and speech recognition program
KR102444061B1 (en) * 2015-11-02 2022-09-16 삼성전자주식회사 Electronic device and method for recognizing voice of speech
CN105609118B (en) * 2015-12-30 2020-02-07 生迪智慧科技有限公司 Voice detection method and device
CN105551491A (en) * 2016-02-15 2016-05-04 海信集团有限公司 Voice recognition method and device
JP6794809B2 (en) * 2016-12-07 2020-12-02 富士通株式会社 Voice processing device, voice processing program and voice processing method
JP6392950B1 (en) * 2017-08-03 2018-09-19 ヤフー株式会社 Detection apparatus, detection method, and detection program
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
CN108877778B (en) * 2018-06-13 2019-09-17 百度在线网络技术（北京）有限公司 Sound end detecting method and equipment
US11227117B2 (en) * 2018-08-03 2022-01-18 International Business Machines Corporation Conversation boundary determination
JP7035979B2 (en) * 2018-11-19 2022-03-15 トヨタ自動車株式会社 Speech recognition device
JP7275711B2 (en) 2019-03-20 2023-05-18 ヤマハ株式会社 How audio signals are processed
CN112259108A (en) * 2020-09-27 2021-01-22 科大讯飞股份有限公司 Engine response time analysis method, electronic device and storage medium
CN113314113B (en) * 2021-05-19 2023-11-28 广州大学 Intelligent socket control method, device, equipment and storage medium
CN114898755B (en) * 2022-07-14 2023-01-17 科大讯飞股份有限公司 Voice processing method and related device, electronic equipment and storage medium

Citations (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4239936A (en) * 1977-12-28 1980-12-16 Nippon Electric Co., Ltd. Speech recognition system
US4531228A (en) * 1981-10-20 1985-07-23 Nissan Motor Company, Limited Speech recognition system for an automotive vehicle
JPS61156100A (en) 1984-12-27 1986-07-15 日本電気株式会社 Voice recognition equipment
JPS62211699A (en) 1986-03-13 1987-09-17 株式会社東芝 Voice section detecting circuit
JPS62237498A (en) 1986-04-08 1987-10-17 沖電気工業株式会社 Voice section detecting method
US4829578A (en) * 1986-10-02 1989-05-09 Dragon Systems, Inc. Speech detection and recognition apparatus for use with background noise of varying levels
JPH03105465A (en) 1989-09-19 1991-05-02 Nec Corp Compound word extraction device
JPH0416999A (en) 1990-05-11 1992-01-21 Seiko Epson Corp Speech recognition device
JPH0458297A (en) 1990-06-27 1992-02-25 Toshiba Corp Sound detecting device
US5201028A (en) * 1990-09-21 1993-04-06 Theis Peter F System for distinguishing or counting spoken itemized expressions
US5293588A (en) 1990-04-09 1994-03-08 Kabushiki Kaisha Toshiba Speech detection apparatus not affected by input energy or background noise levels
JPH08106295A (en) 1994-10-05 1996-04-23 Atr Onsei Honyaku Tsushin Kenkyusho:Kk Method and device for recognizing pattern
US5611019A (en) 1993-05-19 1997-03-11 Matsushita Electric Industrial Co., Ltd. Method and an apparatus for speech detection for determining whether an input signal is speech or nonspeech
US5649055A (en) * 1993-03-26 1997-07-15 Hughes Electronics Voice activity detector for speech signals in variable background noise
JPH09245125A (en) 1996-03-06 1997-09-19 Toshiba Corp Pattern recognition device and dictionary correcting method in the device
JPH10254476A (en) 1997-03-14 1998-09-25 Nippon Telegr & Teleph Corp <Ntt> Voice interval detecting method
JPH1152977A (en) 1997-07-31 1999-02-26 Toshiba Corp Method and device for voice processing
US5991721A (en) 1995-05-31 1999-11-23 Sony Corporation Apparatus and method for processing natural language and apparatus and method for speech recognition
JP2000081893A (en) 1998-09-04 2000-03-21 Matsushita Electric Ind Co Ltd Method of speaker adaptation or speaker normalization
US6161087A (en) * 1998-10-05 2000-12-12 Lernout & Hauspie Speech Products N.V. Speech-recognition-assisted selective suppression of silent and filled speech pauses during playback of an audio recording
US6263309B1 (en) 1998-04-30 2001-07-17 Matsushita Electric Industrial Co., Ltd. Maximum likelihood method for finding an adapted speaker model in eigenvoice space
US6317710B1 (en) * 1998-08-13 2001-11-13 At&T Corp. Multimedia search apparatus and method for searching multimedia content using speaker detection by audio data
US6327565B1 (en) 1998-04-30 2001-12-04 Matsushita Electric Industrial Co., Ltd. Speaker and environment adaptation based on eigenvoices
US20020138254A1 (en) * 1997-07-18 2002-09-26 Takehiko Isaka Method and apparatus for processing speech signals
US6529872B1 (en) 2000-04-18 2003-03-04 Matsushita Electric Industrial Co., Ltd. Method for noise adaptation in automatic speech recognition using transformed matrices
US6600874B1 (en) * 1997-03-19 2003-07-29 Hitachi, Ltd. Method and device for detecting starting and ending points of sound segment in video
JP2003303000A (en) 2002-03-15 2003-10-24 Matsushita Electric Ind Co Ltd Method and apparatus for feature domain joint channel and additive noise compensation
US20040064314A1 (en) 2002-09-27 2004-04-01 Aubert Nicolas De Saint Methods and apparatus for speech end-point detection
US20040102965A1 (en) * 2002-11-21 2004-05-27 Rapoport Ezra J. Determining a pitch period
US6757652B1 (en) * 1998-03-03 2004-06-29 Koninklijke Philips Electronics N.V. Multiple stage speech recognizer
JP2004192603A (en) 2002-07-16 2004-07-08 Nec Corp Method of extracting pattern feature, and device therefor
US20040215458A1 (en) 2003-04-28 2004-10-28 Hajime Kobayashi Voice recognition apparatus, voice recognition method and program for voice recognition
JP2005031632A (en) 2003-06-19 2005-02-03 Advanced Telecommunication Research Institute International Utterance section detecting device, voice energy normalizing device, computer program, and computer
US20060053003A1 (en) 2003-06-11 2006-03-09 Tetsu Suzuki Acoustic interval detection method and device
US20060206330A1 (en) * 2004-12-22 2006-09-14 David Attwater Mode confidence
US20060287859A1 (en) * 2005-06-15 2006-12-21 Harman Becker Automotive Systems-Wavemakers, Inc Speech end-pointer
US20070088548A1 (en) 2005-10-19 2007-04-19 Kabushiki Kaisha Toshiba Device, method, and computer program product for determining speech/non-speech
US7236929B2 (en) * 2001-05-09 2007-06-26 Plantronics, Inc. Echo suppression and speech detection techniques for telephony applications
JP2007233148A (en) 2006-03-02 2007-09-13 Nippon Hoso Kyokai <Nhk> Device and program for utterance section detection
US7634401B2 (en) * 2005-03-09 2009-12-15 Canon Kabushiki Kaisha Speech recognition method for determining missing speech

Patent Citations (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4239936A (en) * 1977-12-28 1980-12-16 Nippon Electric Co., Ltd. Speech recognition system
US4531228A (en) * 1981-10-20 1985-07-23 Nissan Motor Company, Limited Speech recognition system for an automotive vehicle
JPS61156100A (en) 1984-12-27 1986-07-15 日本電気株式会社 Voice recognition equipment
JPS62211699A (en) 1986-03-13 1987-09-17 株式会社東芝 Voice section detecting circuit
JPS62237498A (en) 1986-04-08 1987-10-17 沖電気工業株式会社 Voice section detecting method
US4829578A (en) * 1986-10-02 1989-05-09 Dragon Systems, Inc. Speech detection and recognition apparatus for use with background noise of varying levels
JPH03105465A (en) 1989-09-19 1991-05-02 Nec Corp Compound word extraction device
US5293588A (en) 1990-04-09 1994-03-08 Kabushiki Kaisha Toshiba Speech detection apparatus not affected by input energy or background noise levels
JPH0416999A (en) 1990-05-11 1992-01-21 Seiko Epson Corp Speech recognition device
JPH0458297A (en) 1990-06-27 1992-02-25 Toshiba Corp Sound detecting device
US5201028A (en) * 1990-09-21 1993-04-06 Theis Peter F System for distinguishing or counting spoken itemized expressions
US5649055A (en) * 1993-03-26 1997-07-15 Hughes Electronics Voice activity detector for speech signals in variable background noise
US5611019A (en) 1993-05-19 1997-03-11 Matsushita Electric Industrial Co., Ltd. Method and an apparatus for speech detection for determining whether an input signal is speech or nonspeech
JPH08106295A (en) 1994-10-05 1996-04-23 Atr Onsei Honyaku Tsushin Kenkyusho:Kk Method and device for recognizing pattern
US5754681A (en) 1994-10-05 1998-05-19 Atr Interpreting Telecommunications Research Laboratories Signal pattern recognition apparatus comprising parameter training controller for training feature conversion parameters and discriminant functions
US5991721A (en) 1995-05-31 1999-11-23 Sony Corporation Apparatus and method for processing natural language and apparatus and method for speech recognition
JPH09245125A (en) 1996-03-06 1997-09-19 Toshiba Corp Pattern recognition device and dictionary correcting method in the device
JPH10254476A (en) 1997-03-14 1998-09-25 Nippon Telegr & Teleph Corp <Ntt> Voice interval detecting method
JP3105465B2 (en) 1997-03-14 2000-10-30 日本電信電話株式会社 Voice section detection method
US6600874B1 (en) * 1997-03-19 2003-07-29 Hitachi, Ltd. Method and device for detecting starting and ending points of sound segment in video
US20020138254A1 (en) * 1997-07-18 2002-09-26 Takehiko Isaka Method and apparatus for processing speech signals
JPH1152977A (en) 1997-07-31 1999-02-26 Toshiba Corp Method and device for voice processing
US6757652B1 (en) * 1998-03-03 2004-06-29 Koninklijke Philips Electronics N.V. Multiple stage speech recognizer
US6327565B1 (en) 1998-04-30 2001-12-04 Matsushita Electric Industrial Co., Ltd. Speaker and environment adaptation based on eigenvoices
US6343267B1 (en) 1998-04-30 2002-01-29 Matsushita Electric Industrial Co., Ltd. Dimensionality reduction for speaker normalization and speaker and environment adaptation using eigenvoice techniques
US6263309B1 (en) 1998-04-30 2001-07-17 Matsushita Electric Industrial Co., Ltd. Maximum likelihood method for finding an adapted speaker model in eigenvoice space
US6317710B1 (en) * 1998-08-13 2001-11-13 At&T Corp. Multimedia search apparatus and method for searching multimedia content using speaker detection by audio data
JP2000081893A (en) 1998-09-04 2000-03-21 Matsushita Electric Ind Co Ltd Method of speaker adaptation or speaker normalization
US6161087A (en) * 1998-10-05 2000-12-12 Lernout & Hauspie Speech Products N.V. Speech-recognition-assisted selective suppression of silent and filled speech pauses during playback of an audio recording
US6691091B1 (en) 2000-04-18 2004-02-10 Matsushita Electric Industrial Co., Ltd. Method for additive and convolutional noise adaptation in automatic speech recognition using transformed matrices
US7089182B2 (en) 2000-04-18 2006-08-08 Matsushita Electric Industrial Co., Ltd. Method and apparatus for feature domain joint channel and additive noise compensation
US6529872B1 (en) 2000-04-18 2003-03-04 Matsushita Electric Industrial Co., Ltd. Method for noise adaptation in automatic speech recognition using transformed matrices
US7236929B2 (en) * 2001-05-09 2007-06-26 Plantronics, Inc. Echo suppression and speech detection techniques for telephony applications
JP2003303000A (en) 2002-03-15 2003-10-24 Matsushita Electric Ind Co Ltd Method and apparatus for feature domain joint channel and additive noise compensation
JP2004192603A (en) 2002-07-16 2004-07-08 Nec Corp Method of extracting pattern feature, and device therefor
US20080304750A1 (en) 2002-07-16 2008-12-11 Nec Corporation Pattern feature extraction method and device for the same
US20050201595A1 (en) 2002-07-16 2005-09-15 Nec Corporation Pattern characteristic extraction method and device for the same
US20040064314A1 (en) 2002-09-27 2004-04-01 Aubert Nicolas De Saint Methods and apparatus for speech end-point detection
JP2004272201A (en) 2002-09-27 2004-09-30 Matsushita Electric Ind Co Ltd Method and device for detecting speech end point
US20040102965A1 (en) * 2002-11-21 2004-05-27 Rapoport Ezra J. Determining a pitch period
US20040215458A1 (en) 2003-04-28 2004-10-28 Hajime Kobayashi Voice recognition apparatus, voice recognition method and program for voice recognition
JP2004325979A (en) 2003-04-28 2004-11-18 Pioneer Electronic Corp Speech recognition device, speech recognition method, speech recognition program, and information recording medium
US20060053003A1 (en) 2003-06-11 2006-03-09 Tetsu Suzuki Acoustic interval detection method and device
JP2005031632A (en) 2003-06-19 2005-02-03 Advanced Telecommunication Research Institute International Utterance section detecting device, voice energy normalizing device, computer program, and computer
US20060206330A1 (en) * 2004-12-22 2006-09-14 David Attwater Mode confidence
US7634401B2 (en) * 2005-03-09 2009-12-15 Canon Kabushiki Kaisha Speech recognition method for determining missing speech
US20060287859A1 (en) * 2005-06-15 2006-12-21 Harman Becker Automotive Systems-Wavemakers, Inc Speech end-pointer
US20070088548A1 (en) 2005-10-19 2007-04-19 Kabushiki Kaisha Toshiba Device, method, and computer program product for determining speech/non-speech
JP2007233148A (en) 2006-03-02 2007-09-13 Nippon Hoso Kyokai <Nhk> Device and program for utterance section detection

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
K. Ishii et al, "Easy-to-Understand Pattern Recognition", NTT Communication Science Laboratories, Ohmsha, Ltd. (1998).
N. Binder et al., "Speech Non-Speech Separation With GMMS", Proc. Acoustic Society of Japan Fall Meeting, vol. 1, pp. 141-142 (2001).
Office Action in Japanese Application No. 2006-263113 dated Nov. 11, 2008 and partial English-language translation thereof.
Ponceleon et al., Automatic Discovery of Salient Segments in Imperfect Speech Transcripts, Oct. 2001, ACM, 1-58113-436-3/01/0011.
Yamamoto et al., U.S. Appl. No. 11/582,547, filed Oct. 18, 2006.
Yusuke Kida et al.; "Voice Activity Detection based on Optimally Weighted Combination of Multiple Features"; Information Processing Society of Japan; NII-Electronic Library Service; Jul. 15, 2005; pp. 49-54.

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090254341A1 (en) * 2008-04-03 2009-10-08 Kabushiki Kaisha Toshiba Apparatus, method, and computer program product for judging speech/non-speech
US8380500B2 (en) 2008-04-03 2013-02-19 Kabushiki Kaisha Toshiba Apparatus, method, and computer program product for judging speech/non-speech
US20130282367A1 (en) * 2010-12-24 2013-10-24 Huawei Technologies Co., Ltd. Method and apparatus for performing voice activity detection
US8818811B2 (en) * 2010-12-24 2014-08-26 Huawei Technologies Co., Ltd Method and apparatus for performing voice activity detection
US9390729B2 (en) 2010-12-24 2016-07-12 Huawei Technologies Co., Ltd. Method and apparatus for performing voice activity detection
US9818407B1 (en) * 2013-02-07 2017-11-14 Amazon Technologies, Inc. Distributed endpointing for speech recognition
US9396722B2 (en) * 2013-06-20 2016-07-19 Electronics And Telecommunications Research Institute Method and apparatus for detecting speech endpoint using weighted finite state transducer
US20140379345A1 (en) * 2013-06-20 2014-12-25 Electronic And Telecommunications Research Institute Method and apparatus for detecting speech endpoint using weighted finite state transducer
US10832005B1 (en) 2013-11-21 2020-11-10 Soundhound, Inc. Parsing to determine interruptible state in an utterance by detecting pause duration and complete sentences
US10546576B2 (en) * 2014-04-23 2020-01-28 Google Llc Speech endpointing based on word comparisons
US11004441B2 (en) 2014-04-23 2021-05-11 Google Llc Speech endpointing based on word comparisons
US11636846B2 (en) 2014-04-23 2023-04-25 Google Llc Speech endpointing based on word comparisons
US20180144740A1 (en) * 2016-11-22 2018-05-24 Knowles Electronics, Llc Methods and systems for locating the end of the keyword in voice sensing

Also Published As

Publication number Publication date
JP2008083375A (en) 2008-04-10
US20080077400A1 (en) 2008-03-27
JP4282704B2 (en) 2009-06-24
CN101154378A (en) 2008-04-02

Similar Documents

Publication Publication Date Title
US8099277B2 (en) Speech-duration detector and computer program product therefor
US7756707B2 (en) Signal processing apparatus and method
JP5331784B2 (en) Speech end pointer
US7069221B2 (en) Non-target barge-in detection
US20180293974A1 (en) Spoken language understanding based on buffered keyword spotting and speech recognition
JP4667085B2 (en) Spoken dialogue system, computer program, dialogue control apparatus, and spoken dialogue method
JP6897677B2 (en) Information processing device and information processing method
JP2008256802A (en) Voice recognition device and voice recognition method
US11373635B2 (en) Information processing apparatus that fades system utterance in response to interruption
JP2006208486A (en) Voice inputting device
JP2004109563A (en) Speech interaction system, program for speech interaction, and speech interaction method
US20230223014A1 (en) Adapting Automated Speech Recognition Parameters Based on Hotword Properties
WO2021173220A1 (en) Automated word correction in speech recognition systems
KR20050049207A (en) Dialogue-type continuous speech recognition system and using it endpoint detection method of speech
US6157911A (en) Method and a system for substantially eliminating speech recognition error in detecting repetitive sound elements
WO2017085815A1 (en) Perplexed state determination system, perplexed state determination method, and program
JP4340056B2 (en) Speech recognition apparatus and method
US20240054995A1 (en) Input-aware and input-unaware iterative speech recognition
WO2020203384A1 (en) Volume adjustment device, volume adjustment method, and program
JP4745837B2 (en) Acoustic analysis apparatus, computer program, and speech recognition system
US11195545B2 (en) Method and apparatus for detecting an end of an utterance
JPH09311694A (en) Speech recognition device
US11600273B2 (en) Speech processing apparatus, method, and program
JP6590617B2 (en) Information processing method and apparatus
JP3125928B2 (en) Voice recognition device

Legal Events

Date Code Title Description
AS Assignment

Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMAMOTO, KOICHI;KAWAMURA, AKINORI;REEL/FRAME:019253/0985

Effective date: 20070424

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: TOSHIBA DIGITAL SOLUTIONS CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KABUSHIKI KAISHA TOSHIBA;REEL/FRAME:048547/0187

Effective date: 20190228

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8

AS Assignment

Owner name: TOSHIBA DIGITAL SOLUTIONS CORPORATION, JAPAN

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ADD SECOND RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 48547 FRAME: 187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:KABUSHIKI KAISHA TOSHIBA;REEL/FRAME:050041/0054

Effective date: 20190228

Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ADD SECOND RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 48547 FRAME: 187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:KABUSHIKI KAISHA TOSHIBA;REEL/FRAME:050041/0054

Effective date: 20190228

AS Assignment

Owner name: TOSHIBA DIGITAL SOLUTIONS CORPORATION, JAPAN

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S ADDRESS PREVIOUSLY RECORDED ON REEL 048547 FRAME 0187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KABUSHIKI KAISHA TOSHIBA;REEL/FRAME:052595/0307

Effective date: 20190228

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12