EP0949606A2 - Method and system for speech recognition based on phonetic transcriptions - Google Patents
Method and system for speech recognition based on phonetic transcriptions Download PDFInfo
- Publication number
- EP0949606A2 EP0949606A2 EP99302410A EP99302410A EP0949606A2 EP 0949606 A2 EP0949606 A2 EP 0949606A2 EP 99302410 A EP99302410 A EP 99302410A EP 99302410 A EP99302410 A EP 99302410A EP 0949606 A2 EP0949606 A2 EP 0949606A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- speech
- score
- free
- phone
- predetermined unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
- G10L2015/025—Phonemes, fenemes or fenones being the recognition units
Definitions
- the invention relates generally to a method and apparatus for speech recognition and, more particularly, to a method and system for dynamic speech recognition using free-phone scoring.
- Remote telephone access by a customer to confidential bank or credit card account information has become common.
- the customer enters an account number followed by a Personal Identification Number (PIN) via the telephone keypad.
- PIN Personal Identification Number
- the application automatically accesses the specified account information and compares the entered PIN with that stored in the account. If there is a match between PINs, then the application allows the customer to proceed to access the account information. On the other hand, if the PINs do not match, then the application usually calls for human intervention, such as forwarding the call to an account representative. Often the PINs do not match because the customer has forgotten the PIN, and, therefore, the account representative must request a "secret password," such as the maiden name of the customer's mother, to identify the customer as having authorization to access the account. Such human intervention is costly, requiring a team of representatives waiting to intervene.
- Speech recognition systems are unsuitable for replacing such human intervention.
- Speech recognition systems usually include a database storing voice templates or models, which represent complete words or phrases. The system compares these templates or models, which are constructed from collected data samples, to the received spoken words. Consequently, the database must comprise all possible responses and, therefore, requires the collection and verification of a large number of data samples. Where the recognition system is employed in an application in which customer responses are limited, such a system may be acceptable. Where the recognition system is employed in an application in which customer responses are virtually unlimited, such a system is unacceptable. Thus, a need exists for an improved voice recognition system that does not require the collection and verification of a large number of data samples.
- Bocchieri addresses the problem of requiring data collection and verification to create word templates by allowing a customer to enter anticipated responses into a computer. Once the customer enters anticipated responses via a keyboard, the computer creates a phonetic transcription of each entered word. Creating the phonetic transcription involves accessing a dictionary database, which contains common words and associated phonetic transcriptions, and determining whether the entered word and its associated phonetic transcription already exist. If the phonetic transcription does not exist, the computer proceeds to store the entered word with its associated phonetic transcription in a vocabulary lexicon database.
- the computer Upon receiving a spoken word input, the computer constructs a subword model of the word comprising one or more sequences of subwords. Each subword comprises a series of phonemes. Each phoneme, in turn, represents a discrete sound.
- the computer compares the subword model to the phonetic transcriptions in the vocabulary lexicon database to determine whether the spoken input "matches" the entered anticipated response corresponding to the phonetic transcription.
- the system deems that a match has occurred by assigning a confidence recognition factor to the comparison of the subword model and the phonetic transcription and determining whether that confidence factor exceeds a predetermined confidence threshold value.
- the system cannot recognize spoken data if that data has not been previously entered.
- Bocchieri allows easy customization of the system, the customer of the system must still have prior knowledge of all potentially received spoken data so that it can be entered via the keyboard into the system. Additionally, speech recognition systems must be reliable.
- Some speech recognition systems ensure reliability by taking advantage of these individualistic speech patterns and pronunciations. More specifically, these systems utilize voice transcriptions to create a trained subword model database. This trained subword model database comprises customer-dependent phonemes. While improving reliability, the systems are expensive in set-up and operation.
- Another type of speech recognition system that utilizes a trained subword model involves training only the secret password and the particular phonemes contained therein. This type of system is less costly to implement because a trained subword database containing all phonemes is not necessary. Instead, pre-enrollment involves training only the secret password. Because only the particular secret password is trained, however, any change to the password requires re-enrollment and re-training of the new password. Again, the system is costly to set up and operate. Thus, although systems using trained subword databases are reliable, they are somewhat impractical; the need for an improved voice recognition system, particularly one that does not require prior enrollment, remains unsatisfied.
- a method for recognizing a speech utterance as a predetermined unit of speech comprises generating a free-phone model of the speech utterance and calculating a free-phone score representing the likelihood that the free-phone model accurately represents the speech utterance.
- the method also comprises determining whether the speech utterance matches the predetermined unit of speech based upon its score. In an alternative embodiment, the determination of whether the speech utterance matches the predetermined unit of speech is based upon both a word score and the free-phone score.
- a system for recognizing a speech utterance as a predetermined unit of speech is also provided.
- the speech recognition system 1 comprises a computer 10 having a display 20 and a keyboard 30. More specifically, the computer 10 includes a central processing unit (CPU) 40, a memory 50, a data store 60, and a interface unit 70. The CPU 40, memory 50, datastore 60, and interface unit 70 are connected by data, access, and control lines 80.
- CPU central processing unit
- the memory 50 includes a generate subword spelling program 52, an automatic speech recognition (ASR) algorithm program 54, a discriminating program 56, and a generate free-phone model program 58.
- ASR automatic speech recognition
- these programs are shown and described herein as discrete programs, it is to be understood that such programs need not be so divided. Accordingly, in an alternate embodiment, the functionality of these programs is provided by a single program. Similarly, in another alternate embodiment the functionality of these programs is provided by more than four programs. In general, these programs control the CPU 40 which, in turn, accesses the datastore 60 to receive and store information.
- the datastore 60 comprises two databases that the CPU 40 accesses.
- the datastore 60 includes a customer database 62.
- the customer database 62 includes multiple records, each record containing information pertaining to a particular customer. For example, when the speech recognition system 1 of the present invention is employed to govern access to bank account information, the customer database 62 includes fields containing the customer's account number, customer identifying information, such as a PIN and the maiden name of the customer's mother, and account information, such as available balance. Other information may include amount due, data of last payment, and payment due date.
- the datastore 60 includes a subword model database 64.
- the subword model database 64 stores all possible phonemes. Each phoneme represents a single sound. For example, in the English language, phonemes include “ah” as in “cot,” “ay” as in “hay,” “oo” as in “tool,” “oh” as in “old,” “ch” as in “check,” and “k” as in “cow.”
- phonemes include “ah” as in “cot,” “ay” as in “hay,” “oo” as in “tool,” “oh” as in “old,” “ch” as in “check,” and “k” as in “cow.”
- the generate subword spelling program 52 and the generate free-phone spelling program 58 access the subword model database 64 and use these phonemes to create subword spellings of words.
- the computer 10 includes an interface unit 70.
- the interface unit 70 allows a bank account representative to enter information via the keyboard 30 for storage in the customer database 62.
- the interface unit 70 allows the CPU 40 to display information on the computer display 20.
- the interface unit 70 provides a means through which an external telephone 90 may communicate with the computer 10.
- the telephone 90 is typically operated by a customer calling in from a remote location to access account information stored in the customer database 62.
- the ASR algorithm program 54 is provided is coupled to a second, dedicated CPU.
- the dedicated CPU is coupled to the data, access, and control lines 80 so that the dedicated CPU may receive input and send output to the other components of the system 1.
- the system 1 when a customer accesses the system 1 via the telephone 90, the system 1 requests a particular speech utterance from the customer.
- the speech utterance requested matches particular identifying information stored in the customer database 62.
- the system 1 may request the customer to speak maiden name of the customer's mother, which is stored in the customer database associated with that particular customer.
- the system 1 proceeds to manipulate both the stored data in the customer database 62 and a speech utterance received from a customer in order to determine whether the speech utterance matches the stored information in the customer database 62.
- the system 1 first receives the customer call via the telephone 90. Having received the customer call, the system then requests a claim of identity from the customer. Such a claim of identity typically includes a request for the customer account number. This request can be made automatically by the system 1 and may require only that the customer input the account number via a touch tone telephone pad or spoken input. Thus, no human intervention is necessary.
- the system 1 proceeds to retrieve and access the record in the customer database 62 identified by the received account number.
- the customer database 62 includes a field of information relating to the text string comprising the maiden name of the customer's mother. It is to be understood, however, that the secret password may include any predetermined information, including generally known information like the customer's address, as selected by the service provider.
- the CPU 40 reads the mother's maiden name. The actual name stored in the customer database 62 will have been entered by a bank employee via the keyboard 30 and stored in the database 62 prior to receiving the customer call in step 110. Retrieving and accessing the customer database 62, illustrated as step 230.
- step 230 the system generates a subword spelling of the stored maiden name.
- step 240 the generate subword spelling program 52 causes the CPU 40 to access the subword model database 64. Having read the maiden name from the customer database 62 in step 230, the CPU 40 proceeds to construct a subword model comprising a series of phonemes.
- the system 1 generates the subword spelling of the stored maiden name prior to receiving the customer call identified in step 210.
- the subword spelling is generated upon creation of the customer database 62 and stored as part of the database 62. In this manner, the speed of the system 1 upon receiving the customer call is enhanced.
- the system 1 proceeds to request and receive a speech utterance.
- the particular speech utterance requested depends upon the type of information stored in the customer database 62.
- the system 1 requests the customer to utter their mother's maiden name.
- the system 1 proceeds to generate a free-phone model of that speech utterance.
- Generating the free-phone model is performed by the CPU 40 as controlled by the generate free-phone model program 58.
- the free-phone model is generated independently of the information stored in the customer database 62. In other words, the system 1 dynamically creates the free-phone model based solely on the speech utterance.
- transforming the speech utterance into the free-phone model involves creating a string of phonemes comprising the same number of phonemes as the subword model based upon the text string.
- the speech utterance is received, it is broken down into observation vectors.
- Each observation vector represents a discrete unit of time.
- the CPU 40 by accessing the subword model database 64, compares the frequency information received in each observation vector to known statistical models of the frequency information representing individual phonemes.
- the creation of the free-phone model is inexact.
- the generation of the free-phone model is probabilistic in nature, and there exists a likelihood that the free-phone model exactly represents the actual speech utterance.
- the statistical nature and creation of free-phone models are discussed in greater detail in "A Customer-Configurable System For Voice Label Recognition," by R.C. Rose, E. Lleida, G.W. Erhart, and R.V. Grubbe, herein incorporated by reference. Consequently, the CPU 40 must determine which statistical model most closely matches the received information for a given observation vector.
- the CPU 40 creates the free-phone model by stringing together each of the phonemes that correspond to a matching statistical model. It should be noted, however, that in the present embodiment the free-phone model is restricted in length to the number of phonemes that comprise the text string of the maiden name. Consequently, the free-phone model is the optimal phoneme string of length N, where N is the length of phonemes in the subword spelling of the text string.
- the system 1 obtains a free-phone score based on the likelihood that the free-phone model exactly represents the actual speech utterance. More specifically, the free-phone score represents the overall difference between the information received as the utterance and the information contained in the statistical phoneme models matched to the utterance. Thus, the lower the free-phone score, the more likely the free-phone model accurately represents the speech utterance.
- the system 1 proceeds to calculate a word score in step 280.
- the word score also referred to as a forced-lexicon score, represents the likelihood that the subword spelling of the text string matches the received speech utterance. More specifically, in the present embodiment, the word score represents the sum of the difference between the information contained in each observation vector of the speech utterance and the information of the corresponding statistical phoneme model that comprises the subword spelling of the text string. Thus, the lower the word score, the smaller the difference between the received information and the statistical information, and the greater the likelihood that the speech utterance matches the text string.
- the system determines whether the speech utterance sufficiently matches the stored maiden name to allow the customer access to the information stored in the customer database 62.
- the discrimination algorithm program 56 causes the CPU 40 to determine whether, based upon both the free-phone score and the word score, the received speech utterance matches the stored text string of the maiden name of the customer's mother.
- Step 290 of Figure 2 will now be described in greater detail with reference to Figure 3, and continuing reference to Figure 2.
- the CPU 40 compares the word score to the free-phone score in step 310. If the word score is better than the free-phone score, then in step 320, the CPU 40 determines that the received utterance matches the stored text string. Thus, the customer is authorized to received information concerning the account.
- step 314 the CPU 40 divides the free-phone score by two.
- the CPU 40 proceeds in step 316 to determine whether the word score is better than half of the free-phone score. As indicated in step 318, if the word score is better than half the free-phone score divided by two, then the CPU 40 recognizes the received utterance as matching the stored text string.
- the system 1 provides the customer with access to the information stored in the particular record in the customer database 62.
- step 320 the system 1 determines that no match exists and that the customer is not authorized. Where no match exists, the system 1 preferably repeats the operation from step 250 for a second time, so as to remedy an erroneous denial of access due to an unclear utterance.
- the operation of the system 1 according to Figures 2 and 3 results in approximately a 95% correct acceptance rate and a 0% incorrect acceptance rate.
- the discrimination algorithm program 56 generates a confidence score that represents the absolute value of the difference between the free-phone score and the word score. In such an embodiment, the lower the confidence score, the greater the likelihood that the speech utterance matches the text string. In another alternate embodiment, the confidence recognition score represents a weighted average of the word score and free-phone score. The CPU 40 then compares this confidence recognition score to a predetermined threshold confidence recognition value. If the confidence score is less than the threshold value, then the system 1 determines that a match between the speech utterance and the stored maiden name. Consequently, access to the account information is allowed.
- the system 1 finds no match. Denies the customer access to the account information.
- the system generates only a free-phone model and free-phone score. In one such embodiment, the system determines whether the received utterance matches the stored unit of speech by comparing the free-phone score to a threshold value.
Abstract
Description
- The invention relates generally to a method and apparatus for speech recognition and, more particularly, to a method and system for dynamic speech recognition using free-phone scoring.
- Remote telephone access by a customer to confidential bank or credit card account information has become common. Typically, the customer enters an account number followed by a Personal Identification Number (PIN) via the telephone keypad. The application automatically accesses the specified account information and compares the entered PIN with that stored in the account. If there is a match between PINs, then the application allows the customer to proceed to access the account information. On the other hand, if the PINs do not match, then the application usually calls for human intervention, such as forwarding the call to an account representative. Often the PINs do not match because the customer has forgotten the PIN, and, therefore, the account representative must request a "secret password," such as the maiden name of the customer's mother, to identify the customer as having authorization to access the account. Such human intervention is costly, requiring a team of representatives waiting to intervene.
- Typical speech recognition systems are unsuitable for replacing such human intervention. Speech recognition systems usually include a database storing voice templates or models, which represent complete words or phrases. The system compares these templates or models, which are constructed from collected data samples, to the received spoken words. Consequently, the database must comprise all possible responses and, therefore, requires the collection and verification of a large number of data samples. Where the recognition system is employed in an application in which customer responses are limited, such a system may be acceptable. Where the recognition system is employed in an application in which customer responses are virtually unlimited, such a system is unacceptable. Thus, a need exists for an improved voice recognition system that does not require the collection and verification of a large number of data samples.
- Improvements have been made in the field of speech recognition systems. For example, U.S. Patent No. 5,329,608 to Bocchieri et al. (Bocchieri) is directed to an improved speech recognition system. In general, Bocchieri addresses the problem of requiring data collection and verification to create word templates by allowing a customer to enter anticipated responses into a computer. Once the customer enters anticipated responses via a keyboard, the computer creates a phonetic transcription of each entered word. Creating the phonetic transcription involves accessing a dictionary database, which contains common words and associated phonetic transcriptions, and determining whether the entered word and its associated phonetic transcription already exist. If the phonetic transcription does not exist, the computer proceeds to store the entered word with its associated phonetic transcription in a vocabulary lexicon database.
- Upon receiving a spoken word input, the computer constructs a subword model of the word comprising one or more sequences of subwords. Each subword comprises a series of phonemes. Each phoneme, in turn, represents a discrete sound.
- The computer compares the subword model to the phonetic transcriptions in the vocabulary lexicon database to determine whether the spoken input "matches" the entered anticipated response corresponding to the phonetic transcription. The system deems that a match has occurred by assigning a confidence recognition factor to the comparison of the subword model and the phonetic transcription and determining whether that confidence factor exceeds a predetermined confidence threshold value. However, the system cannot recognize spoken data if that data has not been previously entered. Thus, while Bocchieri allows easy customization of the system, the customer of the system must still have prior knowledge of all potentially received spoken data so that it can be entered via the keyboard into the system. Additionally, speech recognition systems must be reliable. Traditionally, the accuracy of speech recognition systems has been ensured by setting a high confidence threshold when comparing the subword model with the phonetic transcription. Such a high confidence threshold ensures that no erroneous access is allowed; however, often the high threshold causes the system to erroneously find no match and deny access to an authorized customer. Individualistic speech patterns and pronunciations and coarticulation error, which often results in the blending of phonemes, are some of the factors that contribute to these erroneous denials of access. These same factors contribute to erroneous allowance of access.
- Some speech recognition systems ensure reliability by taking advantage of these individualistic speech patterns and pronunciations. More specifically, these systems utilize voice transcriptions to create a trained subword model database. This trained subword model database comprises customer-dependent phonemes. While improving reliability, the systems are expensive in set-up and operation.
- Specifically, providing trained subword model databases requires pre-enrollment. Typically, training requires the customer to recite a few sentences containing words that comprise most, if not all, phonemes. The sentences are broken up into the phonemes for use in the customer-dependent subword database. Because the system involves training most or all phonemes, the system has the advantage that the secret password can be changed without retraining the system. The system, however, has the disadvantage of being costly to set up and operate. The added expense lies not only in the creation of each trained subword database, but also in the creation of the entire speech recognition system because a separate customer-dependent model must be created for each customer.
- Another type of speech recognition system that utilizes a trained subword model involves training only the secret password and the particular phonemes contained therein. This type of system is less costly to implement because a trained subword database containing all phonemes is not necessary. Instead, pre-enrollment involves training only the secret password. Because only the particular secret password is trained, however, any change to the password requires re-enrollment and re-training of the new password. Again, the system is costly to set up and operate. Thus, although systems using trained subword databases are reliable, they are somewhat impractical; the need for an improved voice recognition system, particularly one that does not require prior enrollment, remains unsatisfied.
- These needs are satisfied by a method for recognizing a speech utterance as a predetermined unit of speech. The method comprises generating a free-phone model of the speech utterance and calculating a free-phone score representing the likelihood that the free-phone model accurately represents the speech utterance. The method also comprises determining whether the speech utterance matches the predetermined unit of speech based upon its score. In an alternative embodiment, the determination of whether the speech utterance matches the predetermined unit of speech is based upon both a word score and the free-phone score. A system for recognizing a speech utterance as a predetermined unit of speech is also provided.
-
- Figure 1 is an overall schematic view of a speech recognition system according to one embodiment of the present invention;
- Figure 2 is an overall flow diagram of the operation of the speech recognition system of Figure 1; and
- Figure 3 is a detailed flow diagram of one embodiment
of
step 290 of Figure 2. -
- Although the preferred embodiments discussed below are in the context of governing remote access to account information, it will be apparent to one skilled in the art that the speech recognition system of the present invention is equally applicable to other applications. Certain preferred embodiments of the present invention will now be discussed with reference to the drawings.
- With reference to Figure 1, the physical components of a speech recognition system 1 according to one embodiment the present invention will now be described. Generally, the speech recognition system 1 comprises a
computer 10 having a display 20 and a keyboard 30. More specifically, thecomputer 10 includes a central processing unit (CPU) 40, a memory 50, a data store 60, and a interface unit 70. The CPU 40, memory 50, datastore 60, and interface unit 70 are connected by data, access, andcontrol lines 80. - Several programs for controlling the operation of the CPU 40 are stored in memory 50. The memory 50 includes a generate subword spelling program 52, an automatic speech recognition (ASR) algorithm program 54, a discriminating program 56, and a generate free-phone model program 58. Although these programs are shown and described herein as discrete programs, it is to be understood that such programs need not be so divided. Accordingly, in an alternate embodiment, the functionality of these programs is provided by a single program. Similarly, in another alternate embodiment the functionality of these programs is provided by more than four programs. In general, these programs control the CPU 40 which, in turn, accesses the datastore 60 to receive and store information.
- The datastore 60 comprises two databases that the CPU 40 accesses. First, the datastore 60 includes a customer database 62. The customer database 62 includes multiple records, each record containing information pertaining to a particular customer. For example, when the speech recognition system 1 of the present invention is employed to govern access to bank account information, the customer database 62 includes fields containing the customer's account number, customer identifying information, such as a PIN and the maiden name of the customer's mother, and account information, such as available balance. Other information may include amount due, data of last payment, and payment due date.
- Second, the datastore 60 includes a subword model database 64. The subword model database 64 stores all possible phonemes. Each phoneme represents a single sound. For example, in the English language, phonemes include "ah" as in "cot," "ay" as in "hay," "oo" as in "tool," "oh" as in "old," "ch" as in "check," and "k" as in "cow." As discussed below, the generate subword spelling program 52 and the generate free-phone spelling program 58 access the subword model database 64 and use these phonemes to create subword spellings of words.
- Lastly, the
computer 10 includes an interface unit 70. The interface unit 70 allows a bank account representative to enter information via the keyboard 30 for storage in the customer database 62. Similarly, the interface unit 70 allows the CPU 40 to display information on the computer display 20. Additionally, the interface unit 70 provides a means through which an external telephone 90 may communicate with thecomputer 10. The telephone 90 is typically operated by a customer calling in from a remote location to access account information stored in the customer database 62. - In an alternate embodiment, the ASR algorithm program 54 is provided is coupled to a second, dedicated CPU. The dedicated CPU, in turn, is coupled to the data, access, and
control lines 80 so that the dedicated CPU may receive input and send output to the other components of the system 1. - In general, as discussed in greater detail below, when a customer accesses the system 1 via the telephone 90, the system 1 requests a particular speech utterance from the customer. The speech utterance requested matches particular identifying information stored in the customer database 62. For example, the system 1 may request the customer to speak maiden name of the customer's mother, which is stored in the customer database associated with that particular customer. The system 1 proceeds to manipulate both the stored data in the customer database 62 and a speech utterance received from a customer in order to determine whether the speech utterance matches the stored information in the customer database 62.
- The operation of a system 1 will now be described in greater detail with reference to Figure 2, and continuing reference to Figure 1. As illustrated as
step 210 of Figure 2, the system 1 first receives the customer call via the telephone 90. Having received the customer call, the system then requests a claim of identity from the customer. Such a claim of identity typically includes a request for the customer account number. This request can be made automatically by the system 1 and may require only that the customer input the account number via a touch tone telephone pad or spoken input. Thus, no human intervention is necessary. - Once the system 1 requests the claim of identity, as shown as
step 220, the system 1 proceeds to retrieve and access the record in the customer database 62 identified by the received account number. As noted above, in the present embodiment the customer database 62 includes a field of information relating to the text string comprising the maiden name of the customer's mother. It is to be understood, however, that the secret password may include any predetermined information, including generally known information like the customer's address, as selected by the service provider. The CPU 40 reads the mother's maiden name. The actual name stored in the customer database 62 will have been entered by a bank employee via the keyboard 30 and stored in the database 62 prior to receiving the customer call in step 110. Retrieving and accessing the customer database 62, illustrated asstep 230. - Once the system 1 accesses the customer database 62 in
step 230, the system generates a subword spelling of the stored maiden name. Instep 240, the generate subword spelling program 52 causes the CPU 40 to access the subword model database 64. Having read the maiden name from the customer database 62 instep 230, the CPU 40 proceeds to construct a subword model comprising a series of phonemes. - In one embodiment, the system 1 generates the subword spelling of the stored maiden name prior to receiving the customer call identified in
step 210. The subword spelling is generated upon creation of the customer database 62 and stored as part of the database 62. In this manner, the speed of the system 1 upon receiving the customer call is enhanced. - Looking next to step 250, the system 1 proceeds to request and receive a speech utterance. The particular speech utterance requested depends upon the type of information stored in the customer database 62. In the present embodiment, the system 1 requests the customer to utter their mother's maiden name.
- Once the system 1 receives the speech utterance, as shown as
step 250, the system 1 proceeds to generate a free-phone model of that speech utterance. Generating the free-phone model,step 260, is performed by the CPU 40 as controlled by the generate free-phone model program 58. Other than the number of phonemes in the model (as discussed below), the free-phone model is generated independently of the information stored in the customer database 62. In other words, the system 1 dynamically creates the free-phone model based solely on the speech utterance. - In general, transforming the speech utterance into the free-phone model involves creating a string of phonemes comprising the same number of phonemes as the subword model based upon the text string. When the speech utterance is received, it is broken down into observation vectors. Each observation vector represents a discrete unit of time. The CPU 40, by accessing the subword model database 64, compares the frequency information received in each observation vector to known statistical models of the frequency information representing individual phonemes.
- Due to individualistic speech patterns and pronunciations, which result in the blending of phonemes, the creation of the free-phone model is inexact. In other words, the generation of the free-phone model is probabilistic in nature, and there exists a likelihood that the free-phone model exactly represents the actual speech utterance. The statistical nature and creation of free-phone models are discussed in greater detail in "A Customer-Configurable System For Voice Label Recognition," by R.C. Rose, E. Lleida, G.W. Erhart, and R.V. Grubbe, herein incorporated by reference. Consequently, the CPU 40 must determine which statistical model most closely matches the received information for a given observation vector.
- This comparison is repeated for each observation vector received. The CPU 40 creates the free-phone model by stringing together each of the phonemes that correspond to a matching statistical model. It should be noted, however, that in the present embodiment the free-phone model is restricted in length to the number of phonemes that comprise the text string of the maiden name. Consequently, the free-phone model is the optimal phoneme string of length N, where N is the length of phonemes in the subword spelling of the text string.
- As indicated as
step 160, the system 1 obtains a free-phone score based on the likelihood that the free-phone model exactly represents the actual speech utterance. More specifically, the free-phone score represents the overall difference between the information received as the utterance and the information contained in the statistical phoneme models matched to the utterance. Thus, the lower the free-phone score, the more likely the free-phone model accurately represents the speech utterance. - With a subword spelling of the stored maiden name, the system 1 proceeds to calculate a word score in
step 280. The word score, also referred to as a forced-lexicon score, represents the likelihood that the subword spelling of the text string matches the received speech utterance. More specifically, in the present embodiment, the word score represents the sum of the difference between the information contained in each observation vector of the speech utterance and the information of the corresponding statistical phoneme model that comprises the subword spelling of the text string. Thus, the lower the word score, the smaller the difference between the received information and the statistical information, and the greater the likelihood that the speech utterance matches the text string. - Having thus calculated both a free-phone score and a word score, the system 1, in
step 290, determines whether the speech utterance sufficiently matches the stored maiden name to allow the customer access to the information stored in the customer database 62. Specifically, the discrimination algorithm program 56 causes the CPU 40 to determine whether, based upon both the free-phone score and the word score, the received speech utterance matches the stored text string of the maiden name of the customer's mother. - Step 290 of Figure 2 will now be described in greater detail with reference to Figure 3, and continuing reference to Figure 2. As an initial step in determining whether there is a match between the received utterance and the stored text string. The CPU 40 compares the word score to the free-phone score in
step 310. If the word score is better than the free-phone score, then instep 320, the CPU 40 determines that the received utterance matches the stored text string. Thus, the customer is authorized to received information concerning the account. - On the other hand, if the word score is not better than the free-phone score, then in step 314, the CPU 40 divides the free-phone score by two. The CPU 40 proceeds in step 316 to determine whether the word score is better than half of the free-phone score. As indicated in step 318, if the word score is better than half the free-phone score divided by two, then the CPU 40 recognizes the received utterance as matching the stored text string. Thus the system 1 provides the customer with access to the information stored in the particular record in the customer database 62.
- If the word score is not better than one-half of the free-phone score, then in
step 320, the system 1 determines that no match exists and that the customer is not authorized. Where no match exists, the system 1 preferably repeats the operation fromstep 250 for a second time, so as to remedy an erroneous denial of access due to an unclear utterance. Experiments have shown that the operation of the system 1 according to Figures 2 and 3 results in approximately a 95% correct acceptance rate and a 0% incorrect acceptance rate. - It is to be understood that alternate embodiments in which the word score is compared to some fraction, other than one-half, of the free-phone score are within the scope of the present invention. Additionally, in an alternate embodiment, the determination of whether there is a match between the utterance and text string involves only the comparison of
step 310 of Figure 3. - In an alternate embodiment, the discrimination algorithm program 56 generates a confidence score that represents the absolute value of the difference between the free-phone score and the word score. In such an embodiment, the lower the confidence score, the greater the likelihood that the speech utterance matches the text string. In another alternate embodiment, the confidence recognition score represents a weighted average of the word score and free-phone score. The CPU 40 then compares this confidence recognition score to a predetermined threshold confidence recognition value. If the confidence score is less than the threshold value, then the system 1 determines that a match between the speech utterance and the stored maiden name. Consequently, access to the account information is allowed.
- On the other hand, if the CPU 40 determines that the confidence recognition score is greater than the threshold value, then the system 1 finds no match. Denies the customer access to the account information.
- In an alternative embodiment, the system generates only a free-phone model and free-phone score. In one such embodiment, the system determines whether the received utterance matches the stored unit of speech by comparing the free-phone score to a threshold value.
- Although the present invention has been described in terms of certain preferred embodiment, it is to be understood that other embodiments are equally within the scope of the invention. Accordingly, the scope of the present invention is intended to be limited only by the claims appended hereto.
Claims (13)
- A method for recognizing a speech utterance as a predetermined unit of speech, the method comprising:generating a free-phone model of the speech utterance;calculating a free-phone score, said free-phone score representing a likelihood that said free-phone model accurately represents the speech utterance; anddetermining, based upon said free-phone score, whether the speech utterance matches the predetermined unit of speech.
- The method of claim 1 further comprising the steps:generating a subword spelling of the predetermined unit of speech; andcalculating a word score, said word score representing a likelihood said subword spelling accurately represents the predetermined unit of speech;and wherein said step of determining whether the speech utterance matches the predetermined unit of speech is based upon both said word score and said free-phone score.
- The method of claim 2 wherein said step of generating a subword spelling of the predetermined unit of speech includes accessing a subword model database that stores a plurality of phonemes, each phoneme representing a sound.
- The method of claim 2 or claim 3, wherein said step of determining whether the speech utterance matches the predetermined unit of speech includes:calculating a confidence score based upon said word score and said free-phone score; andcomparing said confidence score to a threshold confidence score.
- The method of claim 4 wherein said step of calculating a confidence score includes calculating a weighted average of said word score and said free-phone score.
- The method of claim 2 or claim 3 wherein said step of determining whether the speech utterance matches the predetermined unit of speech includes comparing said word score to said free-phone score.
- The method of claim 2 or claim 3 wherein said step of determining includes comparing said word score to a fraction of said free-phone score.
- The method of claim 7 wherein said fraction is one half.
- The method of claim 2 or claim 3 wherein said step of determining includes determining a difference between said word score and said free-phone score.
- A speech recognition system for recognizing a speech utterance as a predetermined unit of speech, the system comprising means arranged to carry out a method as claimed in any of the preceding claims.
- An article of manufacture comprising:
a computer readable medium having computer readable program code means embodied thereon, said computer readable program code means comprising means for carrying out a method as claimed in any of claims 1 to 9. - A device comprising:a memory device having a representation of a unit of speech stored therein; anda processor coupled to said memory device, said processor configured to carry out a method as claimed in any of claims 1 to 9, said predetermined unit of speech being the stored unit of speech.
- The device of claim 12 further comprising an input device for receiving said speech utterance, said input device coupled to said processor.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/056,346 US6243678B1 (en) | 1998-04-07 | 1998-04-07 | Method and system for dynamic speech recognition using free-phone scoring |
US56346 | 1998-04-07 |
Publications (3)
Publication Number | Publication Date |
---|---|
EP0949606A2 true EP0949606A2 (en) | 1999-10-13 |
EP0949606A3 EP0949606A3 (en) | 2000-10-11 |
EP0949606B1 EP0949606B1 (en) | 2008-01-02 |
Family
ID=22003805
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP99302410A Expired - Lifetime EP0949606B1 (en) | 1998-04-07 | 1999-03-29 | Method and system for speech recognition based on phonetic transcriptions |
Country Status (4)
Country | Link |
---|---|
US (1) | US6243678B1 (en) |
EP (1) | EP0949606B1 (en) |
JP (1) | JP3703991B2 (en) |
DE (1) | DE69937854T2 (en) |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001249684A (en) * | 2000-03-02 | 2001-09-14 | Sony Corp | Device and method for recognizing speech, and recording medium |
US6957184B2 (en) * | 2000-07-20 | 2005-10-18 | Microsoft Corporation | Context free grammar engine for speech recognition system |
US6931376B2 (en) * | 2000-07-20 | 2005-08-16 | Microsoft Corporation | Speech-related event notification system |
US7139709B2 (en) * | 2000-07-20 | 2006-11-21 | Microsoft Corporation | Middleware layer between speech related applications and engines |
US7451075B2 (en) * | 2000-12-29 | 2008-11-11 | Microsoft Corporation | Compressed speech lexicon and method and apparatus for creating and accessing the speech lexicon |
US7508946B2 (en) * | 2001-06-27 | 2009-03-24 | Sony Corporation | Integrated circuit device, information processing apparatus, memory management method for information storage device, mobile terminal apparatus, semiconductor integrated circuit device, and communication method using mobile terminal apparatus |
DE10147734A1 (en) * | 2001-09-27 | 2003-04-10 | Bosch Gmbh Robert | Method for setting a data structure, in particular phonetic transcriptions for a voice-operated navigation system |
US7203652B1 (en) * | 2002-02-21 | 2007-04-10 | Nuance Communications | Method and system for improving robustness in a speech system |
US7222072B2 (en) * | 2003-02-13 | 2007-05-22 | Sbc Properties, L.P. | Bio-phonetic multi-phrase speaker identity verification |
US20060085189A1 (en) * | 2004-10-15 | 2006-04-20 | Derek Dalrymple | Method and apparatus for server centric speaker authentication |
US20060293898A1 (en) * | 2005-06-22 | 2006-12-28 | Microsoft Corporation | Speech recognition system for secure information |
US8543393B2 (en) * | 2008-05-20 | 2013-09-24 | Calabrio, Inc. | Systems and methods of improving automated speech recognition accuracy using statistical analysis of search terms |
DE102008029264A1 (en) | 2008-06-19 | 2009-12-24 | Schaeffler Kg | Assembly and transport safety arrangement for a switching mechanism of a transmission |
US9659559B2 (en) * | 2009-06-25 | 2017-05-23 | Adacel Systems, Inc. | Phonetic distance measurement system and related methods |
US8306204B2 (en) * | 2010-02-18 | 2012-11-06 | Avaya Inc. | Variable noise control threshold |
US20110224982A1 (en) * | 2010-03-12 | 2011-09-15 | c/o Microsoft Corporation | Automatic speech recognition based upon information retrieval methods |
TWI466101B (en) * | 2012-05-18 | 2014-12-21 | Asustek Comp Inc | Method and system for speech recognition |
CN110459211B (en) | 2018-05-07 | 2023-06-23 | 阿里巴巴集团控股有限公司 | Man-machine conversation method, client, electronic equipment and storage medium |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0562138A1 (en) * | 1992-03-25 | 1993-09-29 | International Business Machines Corporation | Method and apparatus for the automatic generation of Markov models of new words to be added to a speech recognition vocabulary |
US5329608A (en) * | 1992-04-02 | 1994-07-12 | At&T Bell Laboratories | Automatic speech recognizer |
US5581655A (en) * | 1991-01-31 | 1996-12-03 | Sri International | Method for recognizing speech using linguistically-motivated hidden Markov models |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4227177A (en) * | 1978-04-27 | 1980-10-07 | Dialog Systems, Inc. | Continuous speech recognition method |
US4241329A (en) * | 1978-04-27 | 1980-12-23 | Dialog Systems, Inc. | Continuous speech recognition method for improving false alarm rates |
US4481593A (en) * | 1981-10-05 | 1984-11-06 | Exxon Corporation | Continuous speech recognition |
US4489435A (en) * | 1981-10-05 | 1984-12-18 | Exxon Corporation | Method and apparatus for continuous word string recognition |
US4489434A (en) * | 1981-10-05 | 1984-12-18 | Exxon Corporation | Speech recognition method and apparatus |
US4837831A (en) * | 1986-10-15 | 1989-06-06 | Dragon Systems, Inc. | Method for creating and using multiple-word sound models in speech recognition |
US5202952A (en) * | 1990-06-22 | 1993-04-13 | Dragon Systems, Inc. | Large-vocabulary continuous speech prefiltering and processing system |
US5850627A (en) * | 1992-11-13 | 1998-12-15 | Dragon Systems, Inc. | Apparatuses and methods for training and operating speech recognition systems |
US5699456A (en) * | 1994-01-21 | 1997-12-16 | Lucent Technologies Inc. | Large vocabulary connected speech recognition system and method of language representation using evolutional grammar to represent context free grammars |
US5822730A (en) * | 1996-08-22 | 1998-10-13 | Dragon Systems, Inc. | Lexical tree pre-filtering in speech recognition |
US5913192A (en) * | 1997-08-22 | 1999-06-15 | At&T Corp | Speaker identification with user-selected password phrases |
-
1998
- 1998-04-07 US US09/056,346 patent/US6243678B1/en not_active Expired - Lifetime
-
1999
- 1999-03-29 DE DE69937854T patent/DE69937854T2/en not_active Expired - Fee Related
- 1999-03-29 EP EP99302410A patent/EP0949606B1/en not_active Expired - Lifetime
- 1999-04-07 JP JP09951199A patent/JP3703991B2/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5581655A (en) * | 1991-01-31 | 1996-12-03 | Sri International | Method for recognizing speech using linguistically-motivated hidden Markov models |
EP0562138A1 (en) * | 1992-03-25 | 1993-09-29 | International Business Machines Corporation | Method and apparatus for the automatic generation of Markov models of new words to be added to a speech recognition vocabulary |
US5329608A (en) * | 1992-04-02 | 1994-07-12 | At&T Bell Laboratories | Automatic speech recognizer |
Non-Patent Citations (1)
Title |
---|
ROSE R C ET AL: "SPEECH RECOGNTION USING AUTOMATICALLY DERIVED ACOUSTIC BASEFORMS" IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP),US,LOS ALAMITOS, IEEE COMP. SOC. PRESS, 21 April 1997 (1997-04-21), pages 1271-1274, XP000822686 ISBN: 0-8186-7920-4 * |
Also Published As
Publication number | Publication date |
---|---|
JPH11327585A (en) | 1999-11-26 |
DE69937854D1 (en) | 2008-02-14 |
US6243678B1 (en) | 2001-06-05 |
JP3703991B2 (en) | 2005-10-05 |
EP0949606B1 (en) | 2008-01-02 |
DE69937854T2 (en) | 2008-12-24 |
EP0949606A3 (en) | 2000-10-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6243678B1 (en) | Method and system for dynamic speech recognition using free-phone scoring | |
EP0564166B1 (en) | Automatic speech recognizer | |
CN108989341B (en) | Voice autonomous registration method and device, computer equipment and storage medium | |
USRE38101E1 (en) | Methods and apparatus for performing speaker independent recognition of commands in parallel with speaker dependent recognition of names, words or phrases | |
US7421387B2 (en) | Dynamic N-best algorithm to reduce recognition errors | |
US6836760B1 (en) | Use of semantic inference and context-free grammar with speech recognition system | |
US4653097A (en) | Individual verification apparatus | |
US8731928B2 (en) | Speaker adaptation of vocabulary for speech recognition | |
US5384701A (en) | Language translation system | |
US5913192A (en) | Speaker identification with user-selected password phrases | |
US6912499B1 (en) | Method and apparatus for training a multilingual speech model set | |
US4852180A (en) | Speech recognition by acoustic/phonetic system and technique | |
Chou | Discriminant-function-based minimum recognition error rate pattern-recognition approach to speech recognition | |
US9286887B2 (en) | Concise dynamic grammars using N-best selection | |
WO2000003386A1 (en) | Language independent speech recognition | |
CN104299623A (en) | Automated confirmation and disambiguation modules in voice applications | |
US6208965B1 (en) | Method and apparatus for performing a name acquisition based on speech recognition | |
US11204964B1 (en) | Systems and methods for conversing with a user | |
US6963834B2 (en) | Method of speech recognition using empirically determined word candidates | |
US20220180864A1 (en) | Dialogue system, dialogue processing method, translating apparatus, and method of translation | |
US20050187767A1 (en) | Dynamic N-best algorithm to reduce speech recognition errors | |
EP1213706B1 (en) | Method for online adaptation of pronunciation dictionaries | |
JP4245948B2 (en) | Voice authentication apparatus, voice authentication method, and voice authentication program | |
JPS63500126A (en) | speaker verification device | |
EP1158491A2 (en) | Personal data spoken input and retrieval |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): DE FR GB |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
AKX | Designation fees paid |
Free format text: DE FR GB |
|
17P | Request for examination filed |
Effective date: 20061110 |
|
17Q | First examination report despatched |
Effective date: 20070109 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 15/06 20060101AFI20070604BHEP |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): DE FR GB |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 69937854 Country of ref document: DE Date of ref document: 20080214 Kind code of ref document: P |
|
ET | Fr: translation filed | ||
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20081003 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20090325 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20090327 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20090316 Year of fee payment: 11 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20100329 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20101130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20100331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20101001 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20100329 |