US8447044B2 - Adaptive LPC noise reduction system - Google Patents

Adaptive LPC noise reduction system Download PDF

Info

Publication number
US8447044B2
US8447044B2 US11/804,577 US80457707A US8447044B2 US 8447044 B2 US8447044 B2 US 8447044B2 US 80457707 A US80457707 A US 80457707A US 8447044 B2 US8447044 B2 US 8447044B2
Authority
US
United States
Prior art keywords
signal
input signal
audio input
low
sampled audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US11/804,577
Other versions
US20080285773A1 (en
Inventor
Rajeev Nongpiur
Phillip A. Hetherington
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BlackBerry Ltd
8758271 Canada Inc
Original Assignee
QNX Software Systems Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by QNX Software Systems Ltd filed Critical QNX Software Systems Ltd
Priority to US11/804,577 priority Critical patent/US8447044B2/en
Assigned to QNX SOFTWARE SYSTEM (WAVEMAKERS), INC. reassignment QNX SOFTWARE SYSTEM (WAVEMAKERS), INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HETHERINGTON, PHILLIP A, NONGPIUR, RAJEEV
Publication of US20080285773A1 publication Critical patent/US20080285773A1/en
Assigned to JPMORGAN CHASE BANK, N.A. reassignment JPMORGAN CHASE BANK, N.A. SECURITY AGREEMENT Assignors: BECKER SERVICE-UND VERWALTUNG GMBH, CROWN AUDIO, INC., HARMAN BECKER AUTOMOTIVE SYSTEMS (MICHIGAN), INC., HARMAN BECKER AUTOMOTIVE SYSTEMS HOLDING GMBH, HARMAN BECKER AUTOMOTIVE SYSTEMS, INC., HARMAN CONSUMER GROUP, INC., HARMAN DEUTSCHLAND GMBH, HARMAN FINANCIAL GROUP LLC, HARMAN HOLDING GMBH & CO. KG, HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, Harman Music Group, Incorporated, HARMAN SOFTWARE TECHNOLOGY INTERNATIONAL BETEILIGUNGS GMBH, HARMAN SOFTWARE TECHNOLOGY MANAGEMENT GMBH, HBAS INTERNATIONAL GMBH, HBAS MANUFACTURING, INC., INNOVATIVE SYSTEMS GMBH NAVIGATION-MULTIMEDIA, JBL INCORPORATED, LEXICON, INCORPORATED, MARGI SYSTEMS, INC., QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC., QNX SOFTWARE SYSTEMS CANADA CORPORATION, QNX SOFTWARE SYSTEMS CO., QNX SOFTWARE SYSTEMS GMBH, QNX SOFTWARE SYSTEMS GMBH & CO. KG, QNX SOFTWARE SYSTEMS INTERNATIONAL CORPORATION, QNX SOFTWARE SYSTEMS, INC., XS EMBEDDED GMBH (F/K/A HARMAN BECKER MEDIA DRIVE TECHNOLOGY GMBH)
Assigned to HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC., QNX SOFTWARE SYSTEMS GMBH & CO. KG reassignment HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED PARTIAL RELEASE OF SECURITY INTEREST Assignors: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Assigned to QNX SOFTWARE SYSTEMS CO. reassignment QNX SOFTWARE SYSTEMS CO. CONFIRMATORY ASSIGNMENT Assignors: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.
Assigned to QNX SOFTWARE SYSTEMS LIMITED reassignment QNX SOFTWARE SYSTEMS LIMITED CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: QNX SOFTWARE SYSTEMS CO.
Publication of US8447044B2 publication Critical patent/US8447044B2/en
Application granted granted Critical
Assigned to 8758271 CANADA INC. reassignment 8758271 CANADA INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: QNX SOFTWARE SYSTEMS LIMITED
Assigned to 2236008 ONTARIO INC. reassignment 2236008 ONTARIO INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: 8758271 CANADA INC.
Assigned to BLACKBERRY LIMITED reassignment BLACKBERRY LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: 2236008 ONTARIO INC.
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L21/0232Processing in the frequency domain
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/12Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being prediction coefficients

Definitions

  • This disclosure relates to noise suppression.
  • this disclosure relates to reducing low-frequency noise in speech signals.
  • Such systems may include cellular telephones, hands-free systems, transcribers, recording devices and voice recognition systems.
  • the speech signal includes many forms of background noise, including low-frequency noise, which may be present in a vehicle.
  • the background noise may be caused by wind, rain, engine noise, road noise, vibration, blower fans, windshield wipers and other sources.
  • the background noise tends to corrupt the speech signal.
  • the background noise, especially low-frequency noise decreases the intelligibility of the speech signal.
  • Some systems attempt to minimize background noise using fixed filters, such as analog high-pass filters. Other systems attempt to selectively attenuate specific frequency bands.
  • the fixed filters may indiscriminately eliminate desired signal content, and may not adapt to changing amplitude levels. There is a need for a system that reduces low-frequency noise in speech signals in a vehicle.
  • a noise suppression system reduces low-frequency noise in a speech signal using linear predictive coefficients in an adaptive filter.
  • a digital filter may update or adapt a limited set of linear predictive coefficients on a sample-by-sample basis.
  • the linear predictive coefficients may model the human vocal tract.
  • the linear predictive coefficients may be used to provide an error signal based on a difference between the speech signal and a delayed speech signal.
  • the error signal may represent an enhanced speech signal having attenuated and normalized low-frequency noise components.
  • the noise suppression system may establish an attenuated amplitude level, and all low-frequency noise components may be programmed to an attenuated level.
  • the attenuated level may represent a normalized or “flattened” signal level.
  • FIG. 1 shows an adaptive noise reduction system in a vehicle environment.
  • FIG. 2 shows an adaptive noise reduction system
  • FIG. 3 shows an adaptive filter coefficient processor
  • FIG. 4 is a flow diagram showing adaptation of the LPC values.
  • FIG. 5 is a spectrograph showing an unprocessed speech waveform in a lower panel.
  • An upper panel shows the same speech waveform processed by the adaptive noise reduction system.
  • FIG. 6 shows an adaptive noise reduction system having a voice activity detector.
  • FIG. 7 is a spectrograph showing an unprocessed speech waveform in a lower panel.
  • An upper panel shows the same waveform processed by the adaptive noise reduction system having the voice activity detector.
  • FIG. 8 shows an adaptive noise reduction system having a wind buffet detector.
  • FIG. 9 is a spectrograph showing an unprocessed speech waveform in a lower panel.
  • An upper panel shows the same waveform processed by the adaptive noise reduction system having a high-pass and low-pass filter.
  • FIG. 1 shows an adaptive noise reduction system 110 in a vehicle environment 120 .
  • the adaptive noise reduction system 110 may receive speech signals from a device that converts sound into operational signals, such as a microphone 130 in a user system 140 .
  • the user system 140 may be a device that receives speech signals where the fidelity of the speech signal is considered.
  • the user systems 140 may include a cellular telephone 142 , a transcriber 144 , a hands-free system 146 , a voice recognition system 148 , a recording device 150 , a speakerphone or other communication system.
  • the adaptive noise reduction system 110 may be interposed between the microphone 130 and the circuitry of the specific user system 140 , or may be incorporated into the specific user system 140 .
  • the adaptive noise reduction system 110 may be used in a user system where speech signals are processed or transmitted.
  • the respective user systems 140 may receive an output signal 160 from the adaptive noise reduction system 110 .
  • the output signal 160 of the adaptive noise reduction system 110 represents enhanced speech signals having reduced noise levels, where low-frequency noise components have been “flattened.”
  • a flattened signal may have frequency components that have been normalized or reduced in amplitude to some predetermined value across a frequency band of interest. For example, if a speech signal includes low-frequency components (noise) in the zero to about 500 Hz region, the amplitude of each frequency component may be set equal to a predetermined amplitude to reduce the average amplitude of the low-frequency signals.
  • FIG. 2 shows the adaptive noise reduction system 110 , which may include a sampling system 212 .
  • the sampling system 212 may couple the microphone 130 to the adaptive noise reduction system 110 .
  • the sampling system 212 may receive an operational signal from the microphone 130 representing speech, and may convert the signal into digital form at a selected sampling rate.
  • the sampling rate may be selected to capture any desired frequency content. For speech, the sampling rate may be approximately 8 kHz to about 22 kHz.
  • the sampling system 212 may include an analog-to-digital converter (ADC) 214 to convert the analog speech signals from the microphone 130 to sampled digital signals.
  • ADC analog-to-digital converter
  • the sampling system 212 may output a continuous sequence of sampled speech signals x(n) to first delay logic 216 .
  • the first delay logic 216 may delay the sampled speech signal x(n) by one sample, and may feed the delayed speech signal x(n ⁇ 1) to an adaptive filter coefficient processor 218 .
  • the adaptive filter coefficient processor 218 may be implemented in hardware and/or software, and may include a digital signal processor (DSP).
  • the DSP 218 may execute instructions that delay an input signal one or more additional times, track frequency components of a signal, filter a signal, and/or attenuate or boost an amplitude of a signal.
  • the adaptive filter coefficient processor or DSP 218 may be implemented as discrete logic or circuitry, a mix of discrete logic and a processor, or may be distributed over multiple processors or software programs.
  • the adaptive filter coefficient processor 218 may process the continuous stream of speech signals x(n) and produce an estimated signal ⁇ circumflex over (x) ⁇ (n).
  • Summing logic 224 may sum the estimated signal ⁇ circumflex over (x) ⁇ (n) and an inverted sampled speech signal ⁇ x(n) to produce an error signal e(n).
  • the summing logic 224 may include an adder, comparator or other logic and circuitry.
  • the error signal e(n) which may be a difference signal
  • the sampled speech signal x(n) may be inverted prior to the summing operation. In FIG. 2 , an inversion is shown by the minus sign preceding “x(n).”
  • the error signal e(n) may then be used to calculate and adaptively update a plurality of linear predictive coefficient values 324 (LPC values).
  • LPC values linear predictive coefficient values
  • FIG. 3 shows the adaptive filter coefficient processor 218 in greater detail.
  • the adaptive filter coefficient processor 218 may include sequentially coupled delay logic 310 .
  • An output signal 312 of each delay logic 310 may feed the input of the subsequent stage.
  • Multiplier logic 320 may multiply the output signal 312 of each delay logic circuit 310 by the respective LPC value 324 .
  • Summing node logic 330 may sum the output of the respective multipliers 320 to implement a sum of products operation and provide the estimated signal ⁇ circumflex over (x) ⁇ (n).
  • the adaptive filter coefficient processor 218 may include five delay logic blocks 310 , not including the first delay logic circuit 216 .
  • the number of LPC values 324 may be one less than the number of delay circuits. Accordingly, FIG. 3 shows six LPC values 324 corresponding to the five delay logic circuits 310 . This indicates that the adaptive filter coefficient processor 218 shown in FIG. 3 may have a length of six or may be a sixth order filter. However, the adaptive filter coefficient processor 218 may dynamically modify the filter order, and thus the number of LPC values, to adapt to a changing environment.
  • the adaptive filter coefficient processor 218 may be a finite impulse response (FIR) time-domain active filter or another filter.
  • the adaptive filter coefficient processor 218 may use a linear predictive approach to model the vocal tract of a speaker.
  • the LPC values 324 may be updated on a sample-by-sample basis, rather than a block approach. However, in some implementations, a block approach may be used.
  • Some linear predictive coding techniques use a block approach to model the human vocal tract. Such linear predictive coding techniques may attempt to model the human speech to compress and encode the speech to reduce the amount of data transmitted. Rather than transmitting actual processed speech samples, such as digitized speech, some linear predictive systems transmit the coefficients along with limited instructions. The receiving system may then use the transmitted coefficients to synthesize the original speech. Such linear predictive systems may effectively “compress” the speech because the transmitted coefficients represent less data than the actual digitized speech samples.
  • the limited instructions transmitted along with the coefficients may include instructions indicating whether a coefficient corresponds to a voiced or unvoiced sound. However, some linear predictive systems may require about one hundred to about one-hundred and fifty coefficients to accurately model speech and produce realistic sounding speech. Use of an insufficient number of coefficients may result in a “mechanical” sounding voice.
  • Some linear predictive coding systems may use the Levinson-Durbin recursive process to calculate the coefficients on a block-by-block basis. A predetermined number of samples are received before the block is processed. A linear predictive system using the Levinson-Durbin algorithm may require one-hundred coefficients (or more). This may necessitate use of a corresponding block size of equal value, for example, one-hundred samples (or more). Some block approaches provide an “average” for the coefficients based on the entire block, rather than on a per sample basis. Accordingly, inaccuracies may arise due to the variation in the speech sample within the block.
  • the adaptive filter coefficient processor 218 may adaptively calculate the LPC values on a sample-by-sample basis. That is, for each new speech sample, the adaptive filter coefficient processor 218 may update all of the LPC values. Thus, the LPC values may quickly adapt to actual changes in the speech samples. The LPC values calculated on a sample-by-sample basis may be more effective in tracking any rapid variations in the vocal tract compared to the block approach. The adaptive filter coefficient processor 218 may dynamically update the LPC values on a sample-by-sample basis to attempt to minimize the error signal, e(n), which may be fed back to the adaptive filter coefficient processor 218 .
  • the error signal, e(n) may be a difference between the estimated signal ⁇ circumflex over (x) ⁇ (n) and the sampled speech signal x(n), which has been inverted.
  • the error signal e(n) may contain the actual processed speech samples and may represent the output to a subsequent stage. In that regard, the error signal e(n) may not contain the LPC values or coefficients as do the outputs of other predictive systems. Because the error signal e(n) may represent the actual digitized speech sample as processed, it cannot approach zero.
  • the first delay logic 216 in part, and use of a low number of LPC values may prevent the estimated signal ⁇ circumflex over (x) ⁇ (n) from precisely duplicating the sampled speech signal x(n). Accordingly, the value of e(n) may not approach zero.
  • the error signal e(n) may be maintained at a sufficiently high value.
  • the vocal tract is modeled by the LPC values 324 .
  • the adaptive filter coefficient processor 218 models an “envelope” of the speech spectrum. This effectively preserves the speech information in the error signal e(n). Any number of LPC values may be used, and the number of such values (and associated delays) may be changed dynamically. For example, between two and twenty LPC values may be used.
  • the error signal e(n) representing the processed speech signal may be converted back to another format, such as an analog signal format, by a digital-to-analog converter (DAC) 330 .
  • the output of the DAC 330 may provide the processed or enhanced output signal 160 to the user system 140 .
  • An LPC adaptation circuit or logic 340 may minimize the error signal e(n) by minimizing the difference between the estimated signal ⁇ circumflex over (x) ⁇ (n) and the sampled speech signal x(n) based on a least-squares type of process.
  • the LPC adaptation circuit 340 may use other processes, such as recursive least-squares, normalized least mean squares, proportional least mean squares and/or least mean squares. Many other processes may be used to minimize the error signal e(n). Further variations of the minimization may be used to ensure that the output does not diverge.
  • the LPC adaptation logic 340 may adaptively update the LPC values on a sample-by-sample basis.
  • the LPC values may be estimated by solving for a i such that the mean square of the error, e(n), may be minimized.
  • the solution may be expressed as a FIR adaptive filter where x(n) is the desired signal, ⁇ circumflex over (x) ⁇ (n) is the estimated signal, a 1 , a 2 , . . . , a N are the adaptive filter coefficients, and x(n ⁇ i) is the reference signal provided to the adaptive filter.
  • FIG. 4 show the acts 400 that the adaptive coefficient processor 218 may take to update the LPC values.
  • Initial LPC values may first be calculated (Act 410 ).
  • the adaptive coefficient processor 218 may then calculate the estimated signal ⁇ circumflex over (x) ⁇ (n) based on the delayed samples (Act 420 ).
  • the adaptive coefficient processor 218 may then invert the sampled signal to obtain an inverted signal ⁇ x(n) (Act 430 ).
  • the error signal e(n) may be obtained by summing the estimated signal and the inverted signal (Act 440 ).
  • the adaptive coefficient processor 218 may minimize the error signal e(n) using a form of least mean squares to estimate the LPC values (Act 450 ).
  • the LPC values 324 may be updated with the estimated LPC values (Act 460 ) so that the LPC values adapt to a changing input signal.
  • the amplitude of the speech signals 510 , 512 and 514 is assumed to be higher than the amplitude of the noise signal 516 .
  • the upper panel shows the same speech waveforms shown in the lower panel, but processed with the adaptive noise reduction system 110 of FIGS. 1-3 .
  • the upper panel shows that the adaptive noise reduction system 110 has significantly reduced the amount of low-frequency noise 530 . That is, its amplitude of the low-frequency noise 530 has been reduced and normalized or flattened.
  • the LPC values 324 may be updated on a sample-by-sample basis so that the system may adapt quickly to a changing input signal.
  • the adaptive filter coefficient processor 218 may attempt to flatten or normalize the signal across a portion or across the entire frequency spectrum. Because of the way the human brain perceives speech, the low-frequency noise, even if lower in amplitude than the speech signal, tends to mask out the speech, thus degrading its quality.
  • the flatness level may be selected in a way such that the spectral envelope of the speech portion of both the processed and unprocessed signals are at similar levels.
  • the level of the flattened spectrum may also be adjusted to approximate the average of the noise spectrum envelope of the unprocessed signal. Because the adaptive filter coefficient processor 218 may flatten or normalize all components across the entire frequency spectrum, both the low-frequency noise 516 and the speech signals 510 , 512 and 514 may be flattened. Thus, the low-frequency content of the speech signal may be somewhat degraded.
  • the adaptive noise reduction system 110 may select a flattened or attenuated level, for example, ⁇ 12 dB.
  • the amplitude of all signals at a particular time is set to ⁇ 12 dB.
  • higher amplitude noise components at 0 dB may be lower by 12 db (from 0 dB to ⁇ 12 dB), but some lower amplitude noise components at ⁇ 20 dB may be raised in amplitude by 8 dB (from ⁇ 20 dB to ⁇ 12 dB). As shown in the upper panel, the average amplitude of the noise signal 530 has been reduced.
  • the LPC values 324 may adapt to the changing input signal caused by the presence of the speech signals 510 , 512 and 514 . Accordingly, all of the components may be normalized or flattened. This may tend to undesirably raise the weak harmonic components of the speech signals to a higher amplitude level, thereby increasing the noise energy and also changing the format structure of the speech signal.
  • the upper panel shows that weak amplitude harmonic components 534 of the speech signal 510 in the 3500 Hz to 5000 Hz range have been undesirably boosted in amplitude.
  • Such high-frequency harmonic artifacts 534 of the speech signal may have ranged in amplitude from ⁇ 20 db to ⁇ 10 db before processing, for example.
  • the flattening of the spectrum may result in an increase of the above-mentioned level by 10 dB to 12 dB.
  • the overall quality of the speech signal shown in the upper panel is improved due to the reduction of the low-frequency noise signal 530 .
  • the low-frequency components removed or flattened by the adaptive noise reduction system 110 may represent wind, rain, engine noise, road noise, vibration, blower fans, windshield wipers and/or other undesired signals that tend to corrupt the speech signal.
  • Variations in signal amplitude may be effectively handled because the adaptive noise reduction system 110 may continuously adapt to the input signal on a sample-by-sample basis. For example, if the amplitude of the noise signal increases suddenly, the adaptive filter coefficient processor 218 may more aggressively attenuate the noise signal to reduce the high amplitude components and flatten the overall amplitude. For example, when the signal is corrupted with high amplitude, low-frequency noise, the adaptive filter may adapt such that the frequency response of the inverse of the LPC values may correspond to the shape of the noise spectrum. However, filtering the signal using the LPC values, rather than using the inverse of the LPC values, results in flattening the noise spectrum in the signal. For this reason, a fixed or nonadapting filter may not provide a satisfactory response. A fixed or non-adaptive filter may always attenuate an input signal by the same amount, regardless of the amplitude of the input signal.
  • the adaptive noise reduction system 110 may include a decision logic circuit 610 and a voice activity detector (VAD) 612 , shown in FIG. 6 .
  • the VAD 612 may receive the speech signal prior to sampling to determine if a speech signal is present.
  • the VAD 612 may inform the decision logic 610 whether voice activity is present.
  • the VAD 612 may determine voice activity based on an average value of the input signal.
  • the VAD 612 may measure the energy of the envelope of the input signal. When the energy of the envelope exceeds a predetermined value, for example, twice the average background level, the VAD may issue a signal to the decision logic 610 indicating detection of voice activity. Accurate voice detection assumes that the energy of the speech signal is greater than the energy of the background or noise signal.
  • a voice activity detector 612 may halt adaptation of the linear predictive coefficients when a speech signal is detected in the presence of noise. Because the linear predictive coefficients may not be updated during the presence of a speech signal, the digital filter may not adapt to the increased energy level of speech signal. Because adaptation may be halted during this time, the amplitude of speech signal across the frequency spectrum may not normalized or flattened.
  • the decision logic circuit 610 may control the adaptation process of the LPC values 324 .
  • the decision logic circuit 610 may prevent adaptation of the LPC values 324 when the VAD 612 detects speech.
  • the LPC values 324 may be maintained at their prior values when a speech signal is detected.
  • the adaptive filter coefficient processor 218 may not adapt or modify the LPC values 324 during voice detection.
  • the decision logic circuit 610 may permit normal adaptation of the LPC values 324 when the VAD 612 indicates that a speech signal is not present. However, in some specific applications, some limited form of filter adaptation may occur when speech is detected.
  • FIG. 7 is a spectrograph showing a speech waveform in both upper and lower panels.
  • FIG. 7 shows three speech signals 510 , 512 and 514 with noise components 516 .
  • the adaptive noise reduction system 110 adapts and may continuously update the LPC values 324 on a sample-by-sample basis to flatten the signal.
  • the VAD 612 may halt adaptation and modification of the LPC values in some applications.
  • the weak amplitude components 720 of the speech signal 510 in about 3500 Hz to about 5000 Hz range may not be artificially raised. This may prevent formation of the high-frequency speech artifacts 534 shown in FIG. 5 .
  • FIG. 8 shows another aspect of the adaptive noise reduction system 110 , and may include a low-pass filter 810 and a high-pass filter 812 , both coupled to the sampling system 210 .
  • the low-pass filter 810 and the high-pass filter 812 may separate the speech signal x(n) into low-frequency components x L (n) and high-frequency components x H (n) for separate processing. Separate processing of low-frequency and high-frequency components may facilitate suppression of wind buffet components that may contain high-amplitude low-frequency noise components.
  • the quality of the speech signal may be greatly improved by reduction or elimination of the wind buffet signals, even if some desirable low-frequency content of the speech signal may also reduced or removed.
  • the low-pass filter 810 may have a cut-off or cross-over frequency at about 800 Hz so that the first delay logic circuit 216 only receives the low-frequency noise signal x L (n), which is below 800 Hz.
  • the high-pass filter 812 may have a cut-off or cross-over frequency at about 800 Hz so that the filter output summing circuit 848 may receive only the high-frequency signal x H (n), which is above 800 Hz.
  • the low-frequency noise signal x L (n) may contain high-amplitude low-frequency wind buffet components.
  • the low-frequency noise signal x L (n) may be processed by the adaptive filter coefficient processor 218 to flatten the low-frequency components, thus reducing or eliminating wind buffet components.
  • a low-pass gain adjustment circuit 842 may adjust a gain of the error signal e(n) to account for flattening of the signal.
  • the gain adjustment circuit 842 may amplify, attenuate or otherwise modify the error signal e(n) by a variable amount of gain 844 .
  • the gain 844 may be adjusted so that the background noise levels of the low-frequency and high-frequency components at the crossover frequency may be approximately equal.
  • a filter output summing circuit 848 may sum the output of the low-pass gain adjustment circuit 842 and an output x H (n) of the high pass filter 812 .
  • the low-frequency wind buffet signals may be flattened or reduced in amplitude by the adaptive filter coefficient processor 218 on a sample-by-sample basis.
  • the flattened noise spectrum in the low-frequency band provided by the adaptive filter coefficient processor 218 may be at a level that that is much lower than the level of the noise spectrum in the high-frequency band.
  • the signal in the low-frequency band may be multiplied by an estimated gain factor 844 so that the spectral level of the noise in the low- and high-frequency bands are the same.
  • a wind buffet detector 846 shown in dashed lines, may be coupled to a decision logic circuit 850 , also shown in dashed lines.
  • the wind buffet detector may be implemented in a similar manner as the wind buffet detection circuitry described in U.S. Patent Application Publication No. US 2004/0165736.
  • U.S. Patent Application Publication No. US 2004/0165736 is incorporated by reference in its entirety.
  • the wind buffet detector 846 may control the decision logic 850 , and may inhibit adaptation of the LPC values 324 when the wind buffet detector indicates that no wind buffets are present in the speech signal x(n). Conversely, the decision logic circuit 850 may permit normal adaptation of the LPC values 324 when the wind buffet detector 846 indicates that wind buffets are present in the speech signal x(n).
  • the LPC values 324 may be maintained at their prior values when wind buffet activity is not detected. That is, the adaptive filter coefficient processor 218 may not adapt or modify the LPC values 324 absent wind buffets.
  • FIG. 9 is a spectrograph showing a speech waveform in both upper and lower panels.
  • the lower panel shows the speech signal in the presence of high-amplitude low-frequency wind buffet components.
  • the upper panel shows the speech signal processed by the circuitry of FIG. 8 .
  • the amplitude of the wind buffet components has been significantly reduced.
  • the logic, circuitry, and processing described above may be encoded in a computer-readable medium such as a CD/ROM, disk, flash memory, RAM or ROM, an electromagnetic signal, or other machine-readable medium as instructions for execution by a processor.
  • the logic may be implemented as analog or digital logic using hardware, such as one or more integrated circuits (including amplifiers, adders, delays, and filters), or one or more processors executing amplification, adding, delaying, and filtering instructions; or in software in an application programming interface (API) or in a Dynamic Link Library (DLL), functions available in a shared memory or defined as local or remote procedure calls; or as a combination of hardware and software.
  • a computer-readable medium such as a CD/ROM, disk, flash memory, RAM or ROM, an electromagnetic signal, or other machine-readable medium as instructions for execution by a processor.
  • the logic may be implemented as analog or digital logic using hardware, such as one or more integrated circuits (including amplifiers, adders, delays, and filters), or one or more processors
  • the logic may be represented in (e.g., stored on or in) a computer-readable medium, machine-readable medium, propagated-signal medium, and/or signal-bearing medium.
  • the media may comprise any device that contains, stores, communicates, propagates, or transports executable instructions for use by or in connection with an instruction executable system, apparatus, or device.
  • the machine-readable medium may selectively be, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared signal or a semiconductor system, apparatus, device, or propagation medium.
  • a non-exhaustive list of examples of a machine-readable medium includes: a magnetic or optical disk, a volatile memory such as a Random Access Memory “RAM,” a Read-Only Memory “ROM,” an Erasable Programmable Read-Only Memory (i.e., EPROM) or Flash memory, or an optical fiber.
  • a machine-readable medium may also include a tangible medium upon which executable instructions are printed, as the logic may be electronically stored as an image or in another format (e.g., through an optical scan), then compiled, and/or interpreted or otherwise processed. The processed medium may then be stored in a computer and/or machine memory.
  • the systems may include additional or different logic and may be implemented in many different ways.
  • a controller may be implemented as a microprocessor, microcontroller, application specific integrated circuit (ASIC), discrete logic, or a combination of other types of circuits or logic.
  • memories may be DRAM, SRAM, Flash, or other types of memory.
  • Parameters (e.g., conditions and thresholds), and other data structures may be separately stored and managed, may be incorporated into a single memory or database, or may be logically and physically organized in many different ways.
  • Programs and instruction sets may be parts of a single program, separate programs, or distributed across several memories and processors.
  • the systems may be included in a wide variety of electronic devices, including a cellular phone, a headset, a hands-free set, a speakerphone, communication interface, or an infotainment system.

Abstract

A noise suppression system reduces low-frequency noise in a speech signal using linear predictive coefficients in an adaptive filter. A digital filter may update or adapt a limited set of linear predictive coefficients on a sample-by-sample basis. The linear predictive coefficients may be used to provide an error signal based on a difference between the speech signal and a delayed speech signal. The error signal represents an enhanced speech signal having attenuated and normalized low-frequency noise components.

Description

BACKGROUND OF THE INVENTION
1. Technical Field
This disclosure relates to noise suppression. In particular, this disclosure relates to reducing low-frequency noise in speech signals.
2. Related Art
Users access various systems to transmit or process speech signals in a vehicle. Such systems may include cellular telephones, hands-free systems, transcribers, recording devices and voice recognition systems.
The speech signal includes many forms of background noise, including low-frequency noise, which may be present in a vehicle. The background noise may be caused by wind, rain, engine noise, road noise, vibration, blower fans, windshield wipers and other sources. The background noise tends to corrupt the speech signal. The background noise, especially low-frequency noise, decreases the intelligibility of the speech signal.
Some systems attempt to minimize background noise using fixed filters, such as analog high-pass filters. Other systems attempt to selectively attenuate specific frequency bands. The fixed filters may indiscriminately eliminate desired signal content, and may not adapt to changing amplitude levels. There is a need for a system that reduces low-frequency noise in speech signals in a vehicle.
SUMMARY
A noise suppression system reduces low-frequency noise in a speech signal using linear predictive coefficients in an adaptive filter. A digital filter may update or adapt a limited set of linear predictive coefficients on a sample-by-sample basis. The linear predictive coefficients may model the human vocal tract. The linear predictive coefficients may be used to provide an error signal based on a difference between the speech signal and a delayed speech signal. The error signal may represent an enhanced speech signal having attenuated and normalized low-frequency noise components.
Low-frequency noise, even if lower in amplitude than the speech signal, tends to mask or reduce the intelligibility of speech. The noise suppression system may establish an attenuated amplitude level, and all low-frequency noise components may be programmed to an attenuated level. The attenuated level may represent a normalized or “flattened” signal level.
Other systems, methods, features and advantages will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the following claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The system may be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like-referenced numerals designate corresponding parts throughout the different views.
FIG. 1 shows an adaptive noise reduction system in a vehicle environment.
FIG. 2 shows an adaptive noise reduction system.
FIG. 3 shows an adaptive filter coefficient processor.
FIG. 4 is a flow diagram showing adaptation of the LPC values.
FIG. 5 is a spectrograph showing an unprocessed speech waveform in a lower panel. An upper panel shows the same speech waveform processed by the adaptive noise reduction system.
FIG. 6 shows an adaptive noise reduction system having a voice activity detector.
FIG. 7 is a spectrograph showing an unprocessed speech waveform in a lower panel. An upper panel shows the same waveform processed by the adaptive noise reduction system having the voice activity detector.
FIG. 8 shows an adaptive noise reduction system having a wind buffet detector.
FIG. 9 is a spectrograph showing an unprocessed speech waveform in a lower panel. An upper panel shows the same waveform processed by the adaptive noise reduction system having a high-pass and low-pass filter.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
FIG. 1 shows an adaptive noise reduction system 110 in a vehicle environment 120. The adaptive noise reduction system 110 may receive speech signals from a device that converts sound into operational signals, such as a microphone 130 in a user system 140. The user system 140 may be a device that receives speech signals where the fidelity of the speech signal is considered. The user systems 140 may include a cellular telephone 142, a transcriber 144, a hands-free system 146, a voice recognition system 148, a recording device 150, a speakerphone or other communication system. The adaptive noise reduction system 110 may be interposed between the microphone 130 and the circuitry of the specific user system 140, or may be incorporated into the specific user system 140. The adaptive noise reduction system 110 may be used in a user system where speech signals are processed or transmitted. The respective user systems 140 may receive an output signal 160 from the adaptive noise reduction system 110.
The output signal 160 of the adaptive noise reduction system 110 represents enhanced speech signals having reduced noise levels, where low-frequency noise components have been “flattened.” A flattened signal may have frequency components that have been normalized or reduced in amplitude to some predetermined value across a frequency band of interest. For example, if a speech signal includes low-frequency components (noise) in the zero to about 500 Hz region, the amplitude of each frequency component may be set equal to a predetermined amplitude to reduce the average amplitude of the low-frequency signals.
FIG. 2 shows the adaptive noise reduction system 110, which may include a sampling system 212. The sampling system 212 may couple the microphone 130 to the adaptive noise reduction system 110. The sampling system 212 may receive an operational signal from the microphone 130 representing speech, and may convert the signal into digital form at a selected sampling rate. The sampling rate may be selected to capture any desired frequency content. For speech, the sampling rate may be approximately 8 kHz to about 22 kHz. The sampling system 212 may include an analog-to-digital converter (ADC) 214 to convert the analog speech signals from the microphone 130 to sampled digital signals.
The sampling system 212 may output a continuous sequence of sampled speech signals x(n) to first delay logic 216. The first delay logic 216 may delay the sampled speech signal x(n) by one sample, and may feed the delayed speech signal x(n−1) to an adaptive filter coefficient processor 218. The adaptive filter coefficient processor 218 may be implemented in hardware and/or software, and may include a digital signal processor (DSP). The DSP 218 may execute instructions that delay an input signal one or more additional times, track frequency components of a signal, filter a signal, and/or attenuate or boost an amplitude of a signal. Alternatively, the adaptive filter coefficient processor or DSP 218 may be implemented as discrete logic or circuitry, a mix of discrete logic and a processor, or may be distributed over multiple processors or software programs.
The adaptive filter coefficient processor 218 may process the continuous stream of speech signals x(n) and produce an estimated signal {circumflex over (x)}(n). Summing logic 224 may sum the estimated signal {circumflex over (x)}(n) and an inverted sampled speech signal −x(n) to produce an error signal e(n). The summing logic 224 may include an adder, comparator or other logic and circuitry. To provide the error signal e(n), which may be a difference signal, the sampled speech signal x(n) may be inverted prior to the summing operation. In FIG. 2, an inversion is shown by the minus sign preceding “x(n).” The error signal e(n) may then be used to calculate and adaptively update a plurality of linear predictive coefficient values 324 (LPC values).
FIG. 3 shows the adaptive filter coefficient processor 218 in greater detail. The adaptive filter coefficient processor 218 may include sequentially coupled delay logic 310. An output signal 312 of each delay logic 310 may feed the input of the subsequent stage. Multiplier logic 320 may multiply the output signal 312 of each delay logic circuit 310 by the respective LPC value 324. Summing node logic 330 may sum the output of the respective multipliers 320 to implement a sum of products operation and provide the estimated signal {circumflex over (x)}(n).
The adaptive filter coefficient processor 218 may include five delay logic blocks 310, not including the first delay logic circuit 216. The number of LPC values 324 may be one less than the number of delay circuits. Accordingly, FIG. 3 shows six LPC values 324 corresponding to the five delay logic circuits 310. This indicates that the adaptive filter coefficient processor 218 shown in FIG. 3 may have a length of six or may be a sixth order filter. However, the adaptive filter coefficient processor 218 may dynamically modify the filter order, and thus the number of LPC values, to adapt to a changing environment.
The adaptive filter coefficient processor 218 may be a finite impulse response (FIR) time-domain active filter or another filter. The adaptive filter coefficient processor 218 may use a linear predictive approach to model the vocal tract of a speaker. The LPC values 324 may be updated on a sample-by-sample basis, rather than a block approach. However, in some implementations, a block approach may be used.
Some linear predictive coding techniques use a block approach to model the human vocal tract. Such linear predictive coding techniques may attempt to model the human speech to compress and encode the speech to reduce the amount of data transmitted. Rather than transmitting actual processed speech samples, such as digitized speech, some linear predictive systems transmit the coefficients along with limited instructions. The receiving system may then use the transmitted coefficients to synthesize the original speech. Such linear predictive systems may effectively “compress” the speech because the transmitted coefficients represent less data than the actual digitized speech samples. The limited instructions transmitted along with the coefficients may include instructions indicating whether a coefficient corresponds to a voiced or unvoiced sound. However, some linear predictive systems may require about one hundred to about one-hundred and fifty coefficients to accurately model speech and produce realistic sounding speech. Use of an insufficient number of coefficients may result in a “mechanical” sounding voice.
Some linear predictive coding systems may use the Levinson-Durbin recursive process to calculate the coefficients on a block-by-block basis. A predetermined number of samples are received before the block is processed. A linear predictive system using the Levinson-Durbin algorithm may require one-hundred coefficients (or more). This may necessitate use of a corresponding block size of equal value, for example, one-hundred samples (or more). Some block approaches provide an “average” for the coefficients based on the entire block, rather than on a per sample basis. Accordingly, inaccuracies may arise due to the variation in the speech sample within the block.
The adaptive filter coefficient processor 218 may adaptively calculate the LPC values on a sample-by-sample basis. That is, for each new speech sample, the adaptive filter coefficient processor 218 may update all of the LPC values. Thus, the LPC values may quickly adapt to actual changes in the speech samples. The LPC values calculated on a sample-by-sample basis may be more effective in tracking any rapid variations in the vocal tract compared to the block approach. The adaptive filter coefficient processor 218 may dynamically update the LPC values on a sample-by-sample basis to attempt to minimize the error signal, e(n), which may be fed back to the adaptive filter coefficient processor 218.
The error signal, e(n), may be a difference between the estimated signal {circumflex over (x)}(n) and the sampled speech signal x(n), which has been inverted. The error signal e(n) may contain the actual processed speech samples and may represent the output to a subsequent stage. In that regard, the error signal e(n) may not contain the LPC values or coefficients as do the outputs of other predictive systems. Because the error signal e(n) may represent the actual digitized speech sample as processed, it cannot approach zero. The first delay logic 216, in part, and use of a low number of LPC values may prevent the estimated signal {circumflex over (x)}(n) from precisely duplicating the sampled speech signal x(n). Accordingly, the value of e(n) may not approach zero.
Because few LPC values are used, the error signal e(n) may be maintained at a sufficiently high value. Thus, the vocal tract is modeled by the LPC values 324. The adaptive filter coefficient processor 218 models an “envelope” of the speech spectrum. This effectively preserves the speech information in the error signal e(n). Any number of LPC values may be used, and the number of such values (and associated delays) may be changed dynamically. For example, between two and twenty LPC values may be used. The error signal e(n) representing the processed speech signal may be converted back to another format, such as an analog signal format, by a digital-to-analog converter (DAC) 330. The output of the DAC 330 may provide the processed or enhanced output signal 160 to the user system 140.
An LPC adaptation circuit or logic 340 may minimize the error signal e(n) by minimizing the difference between the estimated signal {circumflex over (x)}(n) and the sampled speech signal x(n) based on a least-squares type of process. The LPC adaptation circuit 340 may use other processes, such as recursive least-squares, normalized least mean squares, proportional least mean squares and/or least mean squares. Many other processes may be used to minimize the error signal e(n). Further variations of the minimization may be used to ensure that the output does not diverge.
To minimize the error signal, e(n), the LPC adaptation logic 340 may adaptively update the LPC values on a sample-by-sample basis. The error signal, e(n), is given by the equation:
e(n)={circumflex over (x)}(n)−x(n)  (1)
where:
x ^ ( n ) = i = 1 N a i x ( n - i ) ( 2 )
and where:
a1, a2, . . . , aN are the linear prediction coefficients and N is the LPC order. The LPC values may be estimated by solving for ai such that the mean square of the error, e(n), may be minimized. The solution may be expressed as a FIR adaptive filter where x(n) is the desired signal, {circumflex over (x)}(n) is the estimated signal, a1, a2, . . . , aN are the adaptive filter coefficients, and x(n−i) is the reference signal provided to the adaptive filter.
FIG. 4 show the acts 400 that the adaptive coefficient processor 218 may take to update the LPC values. Initial LPC values may first be calculated (Act 410). The adaptive coefficient processor 218 may then calculate the estimated signal {circumflex over (x)}(n) based on the delayed samples (Act 420). The adaptive coefficient processor 218 may then invert the sampled signal to obtain an inverted signal −x(n) (Act 430). The error signal e(n) may be obtained by summing the estimated signal and the inverted signal (Act 440). The adaptive coefficient processor 218 may minimize the error signal e(n) using a form of least mean squares to estimate the LPC values (Act 450). The LPC values 324 may be updated with the estimated LPC values (Act 460) so that the LPC values adapt to a changing input signal.
FIG. 5 is a spectrograph of a speech waveform in both upper and lower panels. Time is shown on the x-axis, frequency is shown on the y-axis, and amplitude is indicated by the color of the signal (if a color drawing) or by the intensity or grayscale (if a black and white drawing). Both panels show three speech signals. For example, a first speech signal 510 begins at about time=0.5 ms and ends at about time=0.75 ms. A second speech signal 512 begins at about time=0.9 ms and ends at about time=1.15 ms. And a third speech signal 514 begins at about time=1.25 ms and ends at about time=1.5 ms.
The lower panel shows the speech signals 510, 512 and 514 corrupted by low-frequency noise 516 in the about 0-500 Hz frequency range. This appears for the duration of the signals from about time=0 to about time=2 ms. The amplitude of the speech signals 510, 512 and 514 is assumed to be higher than the amplitude of the noise signal 516.
The amplitude of the noise drops to a lower noise level shown by reference numeral 518 during the interval from time=0.0 ms to about time=0.5 ms in the 500-3500 Hz frequency range. The amplitude of the noise drops again to a lower background noise level shown by reference numeral 520 from time=0.0 ms to about time=0.5 ms in the 3500-5000 Hz frequency range. The characteristics of the noise signal 516 beyond time=0.5 ms are not addressed.
The upper panel shows the same speech waveforms shown in the lower panel, but processed with the adaptive noise reduction system 110 of FIGS. 1-3. The upper panel shows that the adaptive noise reduction system 110 has significantly reduced the amount of low-frequency noise 530. That is, its amplitude of the low-frequency noise 530 has been reduced and normalized or flattened.
The LPC values 324 may be updated on a sample-by-sample basis so that the system may adapt quickly to a changing input signal. The adaptive filter coefficient processor 218 may attempt to flatten or normalize the signal across a portion or across the entire frequency spectrum. Because of the way the human brain perceives speech, the low-frequency noise, even if lower in amplitude than the speech signal, tends to mask out the speech, thus degrading its quality.
The flatness level may be selected in a way such that the spectral envelope of the speech portion of both the processed and unprocessed signals are at similar levels. The level of the flattened spectrum may also be adjusted to approximate the average of the noise spectrum envelope of the unprocessed signal. Because the adaptive filter coefficient processor 218 may flatten or normalize all components across the entire frequency spectrum, both the low-frequency noise 516 and the speech signals 510, 512 and 514 may be flattened. Thus, the low-frequency content of the speech signal may be somewhat degraded.
As an example, assume that the noise signal 516 ranges in amplitude from 0 dB to −20 dB. Note also that the noise signal 516 overlaps the speech signals 510, 512 and 514, which speech signals have a higher average amplitude than the noise signal 516. Based on the amplitude of the envelope, the adaptive noise reduction system 110 may select a flattened or attenuated level, for example, −12 dB. Thus, the amplitude of all signals at a particular time is set to −12 dB. Accordingly, higher amplitude noise components at 0 dB may be lower by 12 db (from 0 dB to −12 dB), but some lower amplitude noise components at −20 dB may be raised in amplitude by 8 dB (from −20 dB to −12 dB). As shown in the upper panel, the average amplitude of the noise signal 530 has been reduced.
However, the speech signals 510, 512 and 514, which have a higher average energy level than the noise signal, begins at about time=0.5 ms. The LPC values 324 may adapt to the changing input signal caused by the presence of the speech signals 510, 512 and 514. Accordingly, all of the components may be normalized or flattened. This may tend to undesirably raise the weak harmonic components of the speech signals to a higher amplitude level, thereby increasing the noise energy and also changing the format structure of the speech signal. For example, the upper panel shows that weak amplitude harmonic components 534 of the speech signal 510 in the 3500 Hz to 5000 Hz range have been undesirably boosted in amplitude. Such high-frequency harmonic artifacts 534 of the speech signal may have ranged in amplitude from −20 db to −10 db before processing, for example. However, after processing, the flattening of the spectrum may result in an increase of the above-mentioned level by 10 dB to 12 dB.
The overall quality of the speech signal shown in the upper panel is improved due to the reduction of the low-frequency noise signal 530. The low-frequency components removed or flattened by the adaptive noise reduction system 110 may represent wind, rain, engine noise, road noise, vibration, blower fans, windshield wipers and/or other undesired signals that tend to corrupt the speech signal.
Variations in signal amplitude may be effectively handled because the adaptive noise reduction system 110 may continuously adapt to the input signal on a sample-by-sample basis. For example, if the amplitude of the noise signal increases suddenly, the adaptive filter coefficient processor 218 may more aggressively attenuate the noise signal to reduce the high amplitude components and flatten the overall amplitude. For example, when the signal is corrupted with high amplitude, low-frequency noise, the adaptive filter may adapt such that the frequency response of the inverse of the LPC values may correspond to the shape of the noise spectrum. However, filtering the signal using the LPC values, rather than using the inverse of the LPC values, results in flattening the noise spectrum in the signal. For this reason, a fixed or nonadapting filter may not provide a satisfactory response. A fixed or non-adaptive filter may always attenuate an input signal by the same amount, regardless of the amplitude of the input signal.
To reduce or eliminate the high-frequency harmonic artifacts 534 shown in the upper panel of FIG. 5, the adaptive noise reduction system 110 may include a decision logic circuit 610 and a voice activity detector (VAD) 612, shown in FIG. 6. The VAD 612 may receive the speech signal prior to sampling to determine if a speech signal is present. The VAD 612 may inform the decision logic 610 whether voice activity is present. The VAD 612 may determine voice activity based on an average value of the input signal. The VAD 612 may measure the energy of the envelope of the input signal. When the energy of the envelope exceeds a predetermined value, for example, twice the average background level, the VAD may issue a signal to the decision logic 610 indicating detection of voice activity. Accurate voice detection assumes that the energy of the speech signal is greater than the energy of the background or noise signal.
A voice activity detector 612 may halt adaptation of the linear predictive coefficients when a speech signal is detected in the presence of noise. Because the linear predictive coefficients may not be updated during the presence of a speech signal, the digital filter may not adapt to the increased energy level of speech signal. Because adaptation may be halted during this time, the amplitude of speech signal across the frequency spectrum may not normalized or flattened.
The decision logic circuit 610 may control the adaptation process of the LPC values 324. The decision logic circuit 610 may prevent adaptation of the LPC values 324 when the VAD 612 detects speech. The LPC values 324 may be maintained at their prior values when a speech signal is detected. In certain applications, the adaptive filter coefficient processor 218 may not adapt or modify the LPC values 324 during voice detection. Conversely, the decision logic circuit 610 may permit normal adaptation of the LPC values 324 when the VAD 612 indicates that a speech signal is not present. However, in some specific applications, some limited form of filter adaptation may occur when speech is detected.
FIG. 7 is a spectrograph showing a speech waveform in both upper and lower panels. FIG. 7 shows three speech signals 510, 512 and 514 with noise components 516. During presence of noise 516, for example, from time=0 to about 0.5 ms (710), the adaptive noise reduction system 110 adapts and may continuously update the LPC values 324 on a sample-by-sample basis to flatten the signal. However, when the speech signal 510 is detected, the VAD 612 may halt adaptation and modification of the LPC values in some applications. Because the higher energy of the speech signal cannot influence or cause any changes in the LPC values 324, the weak amplitude components 720 of the speech signal 510 in about 3500 Hz to about 5000 Hz range may not be artificially raised. This may prevent formation of the high-frequency speech artifacts 534 shown in FIG. 5.
Accordingly, throughout an entire speech signal 510 segment, the noise signal 516 may be flattened in accordance with the LPC values in effect prior to the beginning of the speech signal 510. Because adaptation is halted during the speech signal 510 in some applications, the integrity of the speech signal is preserved, while eliminating or reducing the noise signal, as shown by reference numeral 726 in the 0-500 Hz frequency range. Adaptation and updating of the LPC values 324 may again begin when the VAD 612 indicates that the speech signal is no longer present, as shown by reference numeral 730 from time=0.75 ms to about time=0.90 ms.
FIG. 8 shows another aspect of the adaptive noise reduction system 110, and may include a low-pass filter 810 and a high-pass filter 812, both coupled to the sampling system 210. The low-pass filter 810 and the high-pass filter 812 may separate the speech signal x(n) into low-frequency components xL(n) and high-frequency components xH(n) for separate processing. Separate processing of low-frequency and high-frequency components may facilitate suppression of wind buffet components that may contain high-amplitude low-frequency noise components.
Because of the way in which the human brain perceives and processes speech, such low-frequency components, even if lower in amplitude than the speech signal, tend to mask the speech signal. Thus, the quality of the speech signal may be greatly improved by reduction or elimination of the wind buffet signals, even if some desirable low-frequency content of the speech signal may also reduced or removed.
The low-pass filter 810 may have a cut-off or cross-over frequency at about 800 Hz so that the first delay logic circuit 216 only receives the low-frequency noise signal xL(n), which is below 800 Hz. Similarly, the high-pass filter 812 may have a cut-off or cross-over frequency at about 800 Hz so that the filter output summing circuit 848 may receive only the high-frequency signal xH(n), which is above 800 Hz.
The low-frequency noise signal xL(n) may contain high-amplitude low-frequency wind buffet components. The low-frequency noise signal xL(n) may be processed by the adaptive filter coefficient processor 218 to flatten the low-frequency components, thus reducing or eliminating wind buffet components.
A low-pass gain adjustment circuit 842 may adjust a gain of the error signal e(n) to account for flattening of the signal. The gain adjustment circuit 842 may amplify, attenuate or otherwise modify the error signal e(n) by a variable amount of gain 844. The gain 844 may be adjusted so that the background noise levels of the low-frequency and high-frequency components at the crossover frequency may be approximately equal. A filter output summing circuit 848 may sum the output of the low-pass gain adjustment circuit 842 and an output xH(n) of the high pass filter 812. The low-frequency wind buffet signals may be flattened or reduced in amplitude by the adaptive filter coefficient processor 218 on a sample-by-sample basis.
The flattened noise spectrum in the low-frequency band provided by the adaptive filter coefficient processor 218 may be at a level that that is much lower than the level of the noise spectrum in the high-frequency band. Thus, to maintain continuity in the noise spectrum, the signal in the low-frequency band may be multiplied by an estimated gain factor 844 so that the spectral level of the noise in the low- and high-frequency bands are the same.
Alternatively, a wind buffet detector 846, shown in dashed lines, may be coupled to a decision logic circuit 850, also shown in dashed lines. The wind buffet detector may be implemented in a similar manner as the wind buffet detection circuitry described in U.S. Patent Application Publication No. US 2004/0165736. U.S. Patent Application Publication No. US 2004/0165736 is incorporated by reference in its entirety.
The wind buffet detector 846 may control the decision logic 850, and may inhibit adaptation of the LPC values 324 when the wind buffet detector indicates that no wind buffets are present in the speech signal x(n). Conversely, the decision logic circuit 850 may permit normal adaptation of the LPC values 324 when the wind buffet detector 846 indicates that wind buffets are present in the speech signal x(n). The LPC values 324 may be maintained at their prior values when wind buffet activity is not detected. That is, the adaptive filter coefficient processor 218 may not adapt or modify the LPC values 324 absent wind buffets.
FIG. 9 is a spectrograph showing a speech waveform in both upper and lower panels. The lower panel shows the speech signal in the presence of high-amplitude low-frequency wind buffet components. The upper panel shows the speech signal processed by the circuitry of FIG. 8. In FIG. 9, the amplitude of the wind buffet components has been significantly reduced.
The logic, circuitry, and processing described above may be encoded in a computer-readable medium such as a CD/ROM, disk, flash memory, RAM or ROM, an electromagnetic signal, or other machine-readable medium as instructions for execution by a processor. Alternatively or additionally, the logic may be implemented as analog or digital logic using hardware, such as one or more integrated circuits (including amplifiers, adders, delays, and filters), or one or more processors executing amplification, adding, delaying, and filtering instructions; or in software in an application programming interface (API) or in a Dynamic Link Library (DLL), functions available in a shared memory or defined as local or remote procedure calls; or as a combination of hardware and software.
The logic may be represented in (e.g., stored on or in) a computer-readable medium, machine-readable medium, propagated-signal medium, and/or signal-bearing medium. The media may comprise any device that contains, stores, communicates, propagates, or transports executable instructions for use by or in connection with an instruction executable system, apparatus, or device. The machine-readable medium may selectively be, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared signal or a semiconductor system, apparatus, device, or propagation medium. A non-exhaustive list of examples of a machine-readable medium includes: a magnetic or optical disk, a volatile memory such as a Random Access Memory “RAM,” a Read-Only Memory “ROM,” an Erasable Programmable Read-Only Memory (i.e., EPROM) or Flash memory, or an optical fiber. A machine-readable medium may also include a tangible medium upon which executable instructions are printed, as the logic may be electronically stored as an image or in another format (e.g., through an optical scan), then compiled, and/or interpreted or otherwise processed. The processed medium may then be stored in a computer and/or machine memory.
The systems may include additional or different logic and may be implemented in many different ways. A controller may be implemented as a microprocessor, microcontroller, application specific integrated circuit (ASIC), discrete logic, or a combination of other types of circuits or logic. Similarly, memories may be DRAM, SRAM, Flash, or other types of memory. Parameters (e.g., conditions and thresholds), and other data structures may be separately stored and managed, may be incorporated into a single memory or database, or may be logically and physically organized in many different ways. Programs and instruction sets may be parts of a single program, separate programs, or distributed across several memories and processors. The systems may be included in a wide variety of electronic devices, including a cellular phone, a headset, a hands-free set, a speakerphone, communication interface, or an infotainment system.
While various embodiments of the invention have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.

Claims (19)

We claim:
1. A noise suppression system comprising:
a sampling circuit adapted to sample an audio input signal at a predetermined sampling rate;
a low-pass filter coupled with the sampling circuit and configured to pass low-frequency components of the sampled audio input signal;
a plurality of delay circuits configured to sequentially delay the low-frequency components of the sampled audio input signal to provide sequentially delayed signals;
an adaptive processor configured to process the sequentially delayed signals and update a plurality of linear predictive coefficient (LPC) values on a sample-by-sample basis, based on an error signal, where the error signal is based on a difference between an output of the low-pass filter and an output of the adaptive processor; and
a decision logic device coupled with the adaptive processor and configured to inhibit an update of the LPC values applied to a portion of the sampled audio input signal based on a determination that a wind buffet is not present in the portion of the sampled audio input signal;
a high-pass filter coupled with the sampling circuit and configured to pass high-frequency components of the sampled audio input signal; and
an adder configured to sum the error signal and an output of the high-pass filter to generate an output signal.
2. The system of claim 1, further comprising a conversion circuit configured to convert the output signal to an analog signal as an enhanced output signal having reduced low-frequency components.
3. The system of claim 1, where between 2 and 20 LPC values are updated on a sample-by-sample basis.
4. The system of claim 1, where the error signal represents enhanced sampled audio speech.
5. The system of claim 4, where noise components of the enhanced sampled audio speech are normalized in amplitude, and an average amplitude of the noise components is reduced.
6. The system of claim 1, further comprising a voice activity detector coupled with the decision logic device and configured to detect a presence of a speech signal, where the decision logic device is configured to inhibit updating of the LPC values applied to the speech signal in response to the detected presence of the speech signal.
7. The system of claim 6, where the detection of the speech signal is based on an average energy level of the sampled audio input signal.
8. The system of claim 1, where the low-pass filter passes low-frequency components of the sampled audio input signal to the adaptive processor, and blocks higher-frequency components of the sampled audio input signal.
9. The system of claim 8, where the low-frequency components are flattened in amplitude.
10. The system of claim 1, further comprising a wind buffet detector coupled with the decision logic device and configured to detect whether the wind buffet is present in the portion of the sampled audio input signal, where the decision logic device is configured to inhibit adaptation of the LPC values in response to a determination by the wind buffet detector that the wind buffet is not present.
11. A noise suppression system comprising:
a sampling circuit adapted to sample an input signal at a predetermined sampling rate;
a low-pass filter coupled with the sampling circuit and configured to pass low-frequency components of the sampled input signal;
an adaptive processor coupled with the low-pass filter and configured to update a plurality of linear predictive coefficient (LPC) values on a sample-by-sample basis, based on an error signal;
where the error signal is based on a difference between an output of the low-pass filter and an output of the adaptive processor, and where the LPC values are configured to flatten the error signal across a frequency region of interest to provide the error signal as an enhanced speech signal having reduced low-frequency components;
a wind buffet detector configured to detect whether wind buffets are present in the sampled input signal;
a decision logic device coupled with the wind buffet detector and configured to inhibit adaptation of the LPC values in response to a determination by the wind buffet detector that a wind buffet is not present in the sampled input signal;
a high-pass filter coupled with the sampling circuit and configured to pass high-frequency components of the sampled input signal; and
an adder configured to sum the error signal and an output of the high-pass filter to generate an output signal.
12. The system of claim 11, where the adaptive processor loosely models a human vocal tract.
13. The system of claim 11, where the error signal represents enhanced sampled speech.
14. A method for enhancing a signal provided to a user device, the method comprising:
sampling an audio input signal at a predetermined sample rate;
filtering the sampled audio input signal through a low-pass filter to pass low-frequency components of the sampled audio input signal;
delaying the low-frequency components of the sampled audio input signal by multiple levels of delays to provide sequentially delayed signals;
processing the sequentially delayed signals in an adaptive filter;
adaptively updating linear predictive coefficient (LPC) values on a sample-by-sample basis based on an error signal, where the error signal is based on a difference between an output of the low-pass filter and an output of the adaptive filter;
determining whether a portion of the sampled audio input signal includes wind buffets;
inhibiting an update of the LPC values applied to the portion of the sampled audio input signal in response to a determination that the portion of the sampled audio input signal does not include a wind buffet;
filtering the sampled audio input signal through a high-pass filter to pass high-frequency components of the sampled audio input signal; and
adding the error signal and an output of the high-pass filter to generate an output signal.
15. The method according to claim 14 further comprising converting the output signal to an analog signal and outputting the analog signal as an enhanced signal to the user device.
16. The method according to claim 14, where the adaptive filter loosely models a human vocal tract.
17. The method according to claim 14, where the low-pass filter passes low-frequency components of the sampled audio input signal to the adaptive filter, and blocks higher-frequency components of the sampled audio input signal.
18. The method according to claim 17, where the low-frequency components are flattened in amplitude.
19. A non-transitory computer-readable storage medium having processor executable instructions to provide a noise-reduced signal by performing the acts of:
sampling an audio input signal at a predetermined sample rate;
filtering the sampled audio input signal through a low-pass filter to pass low-frequency components of the sampled audio input signal;
delaying the low-frequency components of the sampled audio input signal by multiple levels of delays to provide sequentially delayed signals;
processing the sequentially delayed signals in an adaptive filter;
adaptively updating linear predictive coefficient (LPC) values on a sample-by-sample basis based on an error signal, where the error signal is based on a difference between an output of the low-pass filter and an output of the adaptive filter;
determining whether a portion of the sampled audio input signal includes wind buffets;
inhibiting an update of the LPC values applied to the portion of the sampled audio input signal in response to a determination that the portion of the sampled audio input signal does not include a wind buffet;
filtering the sampled audio input signal through a high-pass filter to pass high-frequency components of the sampled audio input signal; and
adding the error signal and an output of the high-pass filter to generate an output signal.
US11/804,577 2007-05-17 2007-05-17 Adaptive LPC noise reduction system Active 2030-12-05 US8447044B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/804,577 US8447044B2 (en) 2007-05-17 2007-05-17 Adaptive LPC noise reduction system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11/804,577 US8447044B2 (en) 2007-05-17 2007-05-17 Adaptive LPC noise reduction system

Publications (2)

Publication Number Publication Date
US20080285773A1 US20080285773A1 (en) 2008-11-20
US8447044B2 true US8447044B2 (en) 2013-05-21

Family

ID=40027499

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/804,577 Active 2030-12-05 US8447044B2 (en) 2007-05-17 2007-05-17 Adaptive LPC noise reduction system

Country Status (1)

Country Link
US (1) US8447044B2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10186260B2 (en) 2017-05-31 2019-01-22 Ford Global Technologies, Llc Systems and methods for vehicle automatic speech recognition error detection
US10462567B2 (en) 2016-10-11 2019-10-29 Ford Global Technologies, Llc Responding to HVAC-induced vehicle microphone buffeting
US10479300B2 (en) 2017-10-06 2019-11-19 Ford Global Technologies, Llc Monitoring of vehicle window vibrations for voice-command recognition
US10525921B2 (en) 2017-08-10 2020-01-07 Ford Global Technologies, Llc Monitoring windshield vibrations for vehicle collision detection
US10562449B2 (en) 2017-09-25 2020-02-18 Ford Global Technologies, Llc Accelerometer-based external sound monitoring during low speed maneuvers

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080312916A1 (en) * 2007-06-15 2008-12-18 Mr. Alon Konchitsky Receiver Intelligibility Enhancement System
US9343079B2 (en) * 2007-06-15 2016-05-17 Alon Konchitsky Receiver intelligibility enhancement system
US8868417B2 (en) * 2007-06-15 2014-10-21 Alon Konchitsky Handset intelligibility enhancement system using adaptive filters and signal buffers
CN102725715B (en) * 2009-10-20 2016-11-09 谱瑞科技股份有限公司 The method and apparatus of the coupled noise effects in minimizing touch control screen controller
EP2647223B1 (en) * 2010-11-29 2019-08-07 Nuance Communications, Inc. Dynamic microphone signal mixer
CN102934061B (en) 2011-02-07 2016-11-09 谱瑞科技股份有限公司 Noise filter means, system and method for capacitance sensing apparatus
US9323385B2 (en) 2011-04-05 2016-04-26 Parade Technologies, Ltd. Noise detection for a capacitance sensing panel
US9170322B1 (en) 2011-04-05 2015-10-27 Parade Technologies, Ltd. Method and apparatus for automating noise reduction tuning in real time
EP2826521B1 (en) * 2013-07-15 2019-12-11 Oticon Medical A/S A hearing assistance device comprising an implanted part for measuring and processing electrically evoked nerve responses
WO2015065362A1 (en) 2013-10-30 2015-05-07 Nuance Communications, Inc Methods and apparatus for selective microphone signal combining

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4243935A (en) * 1979-05-18 1981-01-06 The United States Of America As Represented By The Secretary Of The Navy Adaptive detector
US5208837A (en) * 1990-08-31 1993-05-04 Allied-Signal Inc. Stationary interference cancellor
US5548681A (en) * 1991-08-13 1996-08-20 Kabushiki Kaisha Toshiba Speech dialogue system for realizing improved communication between user and system
US5704000A (en) * 1994-11-10 1997-12-30 Hughes Electronics Robust pitch estimation method and device for telephone speech
JPH1023590A (en) * 1996-07-03 1998-01-23 Matsushita Electric Ind Co Ltd Microphone device
US6230123B1 (en) * 1997-12-05 2001-05-08 Telefonaktiebolaget Lm Ericsson Publ Noise reduction method and apparatus
US20040165736A1 (en) 2003-02-21 2004-08-26 Phil Hetherington Method and apparatus for suppressing wind noise
US6937980B2 (en) * 2001-10-02 2005-08-30 Telefonaktiebolaget Lm Ericsson (Publ) Speech recognition using microphone antenna array
US20060095256A1 (en) * 2004-10-26 2006-05-04 Rajeev Nongpiur Adaptive filter pitch extraction
US20060217976A1 (en) * 2005-03-24 2006-09-28 Mindspeed Technologies, Inc. Adaptive noise state update for a voice activity detector
US7146013B1 (en) * 1999-04-28 2006-12-05 Alpine Electronics, Inc. Microphone system
US7174022B1 (en) * 2002-11-15 2007-02-06 Fortemedia, Inc. Small array microphone for beam-forming and noise suppression

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4243935A (en) * 1979-05-18 1981-01-06 The United States Of America As Represented By The Secretary Of The Navy Adaptive detector
US5208837A (en) * 1990-08-31 1993-05-04 Allied-Signal Inc. Stationary interference cancellor
US5548681A (en) * 1991-08-13 1996-08-20 Kabushiki Kaisha Toshiba Speech dialogue system for realizing improved communication between user and system
US5704000A (en) * 1994-11-10 1997-12-30 Hughes Electronics Robust pitch estimation method and device for telephone speech
JPH1023590A (en) * 1996-07-03 1998-01-23 Matsushita Electric Ind Co Ltd Microphone device
US6230123B1 (en) * 1997-12-05 2001-05-08 Telefonaktiebolaget Lm Ericsson Publ Noise reduction method and apparatus
US7146013B1 (en) * 1999-04-28 2006-12-05 Alpine Electronics, Inc. Microphone system
US6937980B2 (en) * 2001-10-02 2005-08-30 Telefonaktiebolaget Lm Ericsson (Publ) Speech recognition using microphone antenna array
US7174022B1 (en) * 2002-11-15 2007-02-06 Fortemedia, Inc. Small array microphone for beam-forming and noise suppression
US20040165736A1 (en) 2003-02-21 2004-08-26 Phil Hetherington Method and apparatus for suppressing wind noise
US20060095256A1 (en) * 2004-10-26 2006-05-04 Rajeev Nongpiur Adaptive filter pitch extraction
US20060217976A1 (en) * 2005-03-24 2006-09-28 Mindspeed Technologies, Inc. Adaptive noise state update for a voice activity detector

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Linear Predictive Coding by Jeremy Bradbury, dated Dec. 5, 2000.

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10462567B2 (en) 2016-10-11 2019-10-29 Ford Global Technologies, Llc Responding to HVAC-induced vehicle microphone buffeting
US10186260B2 (en) 2017-05-31 2019-01-22 Ford Global Technologies, Llc Systems and methods for vehicle automatic speech recognition error detection
US10525921B2 (en) 2017-08-10 2020-01-07 Ford Global Technologies, Llc Monitoring windshield vibrations for vehicle collision detection
US10562449B2 (en) 2017-09-25 2020-02-18 Ford Global Technologies, Llc Accelerometer-based external sound monitoring during low speed maneuvers
US10479300B2 (en) 2017-10-06 2019-11-19 Ford Global Technologies, Llc Monitoring of vehicle window vibrations for voice-command recognition

Also Published As

Publication number Publication date
US20080285773A1 (en) 2008-11-20

Similar Documents

Publication Publication Date Title
US8447044B2 (en) Adaptive LPC noise reduction system
US8170879B2 (en) Periodic signal enhancement system
US7610196B2 (en) Periodic signal enhancement system
US7680652B2 (en) Periodic signal enhancement system
US8150682B2 (en) Adaptive filter pitch extraction
CA2571417C (en) Advanced periodic signal enhancement
US8326616B2 (en) Dynamic noise reduction using linear model fitting
US8296136B2 (en) Dynamic controller for improving speech intelligibility
US8189810B2 (en) System for processing microphone signals to provide an output signal with reduced interference
US9992572B2 (en) Dereverberation system for use in a signal processing apparatus
US8352257B2 (en) Spectro-temporal varying approach for speech enhancement
EP2244254B1 (en) Ambient noise compensation system robust to high excitation noise
US8306821B2 (en) Sub-band periodic signal enhancement system
US8543390B2 (en) Multi-channel periodic signal enhancement system
US8843367B2 (en) Adaptive equalization system
US8243955B2 (en) System for attenuating noise in an input signal
EP3830823B1 (en) Forced gap insertion for pervasive listening
EP2660814B1 (en) Adaptive equalization system
KR100754558B1 (en) Periodic signal enhancement system

Legal Events

Date Code Title Description
AS Assignment

Owner name: QNX SOFTWARE SYSTEM (WAVEMAKERS), INC., CANADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NONGPIUR, RAJEEV;HETHERINGTON, PHILLIP A;REEL/FRAME:019820/0589

Effective date: 20070808

AS Assignment

Owner name: JPMORGAN CHASE BANK, N.A., NEW YORK

Free format text: SECURITY AGREEMENT;ASSIGNORS:HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED;BECKER SERVICE-UND VERWALTUNG GMBH;CROWN AUDIO, INC.;AND OTHERS;REEL/FRAME:022659/0743

Effective date: 20090331

Owner name: JPMORGAN CHASE BANK, N.A.,NEW YORK

Free format text: SECURITY AGREEMENT;ASSIGNORS:HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED;BECKER SERVICE-UND VERWALTUNG GMBH;CROWN AUDIO, INC.;AND OTHERS;REEL/FRAME:022659/0743

Effective date: 20090331

AS Assignment

Owner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED,CONN

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

Owner name: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.,CANADA

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

Owner name: QNX SOFTWARE SYSTEMS GMBH & CO. KG,GERMANY

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

Owner name: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED, CON

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

Owner name: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC., CANADA

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

Owner name: QNX SOFTWARE SYSTEMS GMBH & CO. KG, GERMANY

Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:024483/0045

Effective date: 20100601

AS Assignment

Owner name: QNX SOFTWARE SYSTEMS CO., CANADA

Free format text: CONFIRMATORY ASSIGNMENT;ASSIGNOR:QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.;REEL/FRAME:024659/0370

Effective date: 20100527

AS Assignment

Owner name: QNX SOFTWARE SYSTEMS LIMITED, CANADA

Free format text: CHANGE OF NAME;ASSIGNOR:QNX SOFTWARE SYSTEMS CO.;REEL/FRAME:027768/0863

Effective date: 20120217

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: 2236008 ONTARIO INC., ONTARIO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:8758271 CANADA INC.;REEL/FRAME:032607/0674

Effective date: 20140403

Owner name: 8758271 CANADA INC., ONTARIO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:QNX SOFTWARE SYSTEMS LIMITED;REEL/FRAME:032607/0943

Effective date: 20140403

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: BLACKBERRY LIMITED, ONTARIO

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:2236008 ONTARIO INC.;REEL/FRAME:053313/0315

Effective date: 20200221

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8