WO2010072245A1

WO2010072245A1 - A method of operating a hearing instrument based on an estimation of present cognitive load of a user and a hearing aid system

Info

Publication number: WO2010072245A1
Application number: PCT/EP2008/068139
Authority: WO
Inventors: Thomas Lunner
Original assignee: Oticon A/S
Priority date: 2008-12-22
Filing date: 2008-12-22
Publication date: 2010-07-01
Also published as: DK3310076T3; DK2914019T3

Abstract

The invention relates to a method of operating a hearing instrument for processing an input sound and to provide an output stimulus according to a user's particular needs. The invention further relates to a system, a computer readable medium and a data processing system. The object of the present invention is to provide an improved customization of a hearing instrument. The problem is solved in that the method comprises the steps a) providing an estimate of the present cognitive load of the user; b) providing processing of an input signal originating from the input sound according to a user's particular needs; and c) adapting the processing in dependence of the estimate the present cognitive load of the user. This has the advantage that the functionality of the hearing aid system is adapted to the current mental state of the user. The estimate of the present cognitive load of a user is produced by in-situ direct measures of cognitive load (e.g. based on EEG-measurements, body temperature, etc.) or by an on-line cognitive model in the hearing aid system whose parameters have been preferably adjusted to fit to the individual user. The invention may e.g. be used in applications where a hearing impaired user's current mental resources are challenged.

Description

A METHOD OF OPERATING A HEARING INSTRUMENT BASED ON AN ESTIMATION OF PRESENT COGNITIVE LOAD OF A USER AND A

HEARING AID SYSTEM

TECHNICAL FIELD

The present invention relates to hearing aids in particular to customization of hearing aids to a user's specific needs. The invention relates specifically to a method of operating a hearing instrument for processing an input sound and to provide an output stimulus according to a user's particular needs.

The invention furthermore relates to a hearing aid system for processing an input sound and to provide an output stimulus according to a user's particular needs.

The invention furthermore relates to a tangible computer-readable medium storing a computer program, and to a data processing system.

The invention may e.g. be useful in applications where a hearing impaired user's current mental resources are challenged.

BACKGROUND ART

The background of the invention is described in two parts:

1. Effects of working memory and cognitive load in difficult listening situations is reviewed

2. Hearing aid signal processing that may improve/ameliorate cognitive load is reviewed

1. Effects of working memory and cognitive load in difficult listening situations

In an optimum listening situation, the speech signal is processed effortlessly and automatically. This means that the cognitive processing involved is largely unconscious and implicit. However, listening conditions are often suboptimum, which means that implicit cognitive processes may be insufficient to unlock the meaning in the speech stream. Resolving ambiguities among previous speech elements and constructing expectations of prospective exchanges in the dialogue are examples of the complex processes that may arise. These processes are effortful and conscious and thus involve explicit cognitive processing.

Working memory (WM) capacity is relatively constant but varies between individuals (Engle et al ., 1999). In performing dual tasks which tax the working memory, there are large individual differences in the ability to assign cognitive resources to both tasks (Li et al., 2001 ). It has yet to be investigated how persons with HI allocate their cognitive resources to different aspects of the language understanding process and how much cognitive spare capacity (CSC) remains to be devoted to other tasks once successful listening has been accomplished.

The ELU (Rόnnberg, 2003; Rόnnberg, Rudner, Foo & Lunner, 2008) relies on the quality of phonological representations in long-term memory, lexical access speed, and explicit storage and processing capacity in working memory. When phonological information extracted from the speech signal can be matched rapidly and smoothly in working memory to phonological representations in long term memory, cognitive processing is implicit and ELU is high. The ELU framework predicts that when mismatch occurs in a communicative situation, it not only elicits a measurable physiological response, it also leads to an engagement of explicit cognitive processes, such as comparison, manipulation and inference making. These processes engage explicit processing and short-term storage capacity in working memory, which can be termed complex working memory capacity. Thus, individual complex working memory capacity is crucial for compensating mismatch.

Listening situations with various background noises or reverberation makes the (speech) signal suboptimal and influence speech recognition both for normal hearing persons and hearing impaired persons but to different extent. Results by Lunner and Sundewall-Thoren (2007) suggests that in an aided condition with slow-acting compression and unmodulated noise the test subjects' cognitive capacities are active, but without exceeding the capacity limit of most hearing impaired individual listeners. Thus, the individual peripheral hearing loss restrains the performance and the performance may be explained by audibility. Possession of greater cognitive capacity confers relatively little benefit. However, in the complex situation with fast-acting compression and varying background noise, much more cognitive capacity is required for successful listening. Thus, the individual cognitive capacity restrains the performance and the speech-in-noise performance may, at least partly, be explained from individual working memory capacity.

Furthermore, Sampralis et al. (2008) have shown that the about 4 dB SNR improvement (attenuation of spatially separated disturbing sources) of directional microphones (in comparison to omnidirectional microphones) have implications for improved memory (recall) and faster response times. Sampralis et al. (2008) have also shown positive results on memory (recall) and response times for noise reduction systems.

A hearing impairment will restrict the amount of information transferred to the brain as well the signal information being of poorer quality compared to normal hearing people because of the perceptual consequences of the cochlear damage, such as reduced time and frequency resolution, difficulties to utilize temporal fine-structure, worse ability for grouping of sound streams as well as worse abilities to segregate sound streams. Thus, for the hearing impaired more situations will provoke effortful explicit processing . For example, hearing impaired are more susceptible to reverberation, background noises, especially fluctuation noises or other talkers, as well as have worse abilities for spatial separation than normal hearing persons.

2. Hearing aid signal processing that may improve/ameliorate cognitive load

Hearing aids have several purposes; first of all they compensate the reduced sensitivity for weak sounds as well as the abnormal growth of loudness through the use of multi-channel compression amplification systems, with either fast or slow time constants (Fast-acting compression can actually be seen as a noise-reduction system under certain conditions, see e.g. Naylor et al. (2006). In addition there are 'helping systems' that may reduce cognitive load that are used in certain situations to improve speech recognition in noise and under other circumstances to increase comfort when speech not is present. Edwards et al. (2007) have shown that directional microphones and noise reduction systems increase memory and reduce response times compared to the unprocessed cases, i.e. indications on less cognitive load. The main components of such helping systems are directional microphones and noise reduction systems. The helping systems are usually automatically invoked based on information from detectors, such as speech/no-speech detectors, signal-to-noise ratio detectors, front/back detectors, and level detectors. The underlying assumption is that the detectors can help to distinguish between 'easier' listening situations and more 'difficultVdemanding situations. This information is used to automate the switching in-and out of the helping systems to help the user to have a comfortable monitoring sound processing when speech is not present to a more aggressive directional microphone set-up and noise reduction system when being in a demanding communication situation.

The 'helping systems' are only used in certain listening situations because they give benefit in only these situations, in other situations they may actually be contra-productive, for example invoking directional microphones, which attenuates sounds from other directions than the frontal direction, in a situation where there are little background noise and/or where information from behind are of importance, the directional microphones may actually worsen for example localization and probably be more effortful than a omnidirectional microphone. Thus, the directional system may negatively influence naturalness, orientation abilities, and object formation, localization abilities.

Similar drawbacks are present for noise reduction systems. DISCLOSURE OF INVENTION

The decision to invoke such helping systems may be dependent on the hearing aid user's cognitive status. For example the correlation between working memory (WM) performance and speech reception threshold (SRT) in noise, as shown in Lunner (2003) and Foo et al. (2007), indicates that people with high WM capacity are more noise tolerant than people with low

WM capacity. This indicates that people with high WM should probably not have the same (SNR) threshold, e.g. when the directional microphone systems or noise reduction systems become active.

Furthermore, what is a demanding situation for one person can be an 'easy' situation for another person depending on their working memory capacity.

And, this is the main point here, when the situation becomes highly dependent on (individual) explicit processing there would probably be a need to switch to the helping systems to be able to manage the situation.

Furthermore, in the future we will see even more aggressive noise reduction systems such as time-frequency masking (Wang et al., 2008) or speech enhancement systems (e.g. Hendriks et al., 2005) as well as aggressive directional systems that are very helpful in certain situations while contra- productive in other situations. Therefore, there will be a need to individually determine when and under which circumstances to shift to the helping systems.

An object of the present invention is to provide an improved customization of a hearing instrument.

Objects of the invention are achieved by the invention described in the accompanying claims and as described in the following. A method

An object of the invention is achieved by a method of operating a hearing instrument for processing an input sound and to provide an output stimulus according to a user's particular needs. The method comprises a) providing an estimate of the present cognitive load of the user; b) providing processing of an input signal originating from the input sound according to a user's particular needs, c) adapting the processing in dependence of the estimate the present cognitive load of the user.

This has the advantage that the functionality of the hearing aid system is adapted to the current mental state of the user.

The invention solves the above problem by utilising direct measures of cognitive load or estimations of cognitive load from an on-line cognitive model in the hearing aid whose parameters have been adjusted to fit to the individual user. When the direct measures of cognitive load indicate high load or that the cognitive model predicts that the cognitive limit of the current user have been exceeded , hel ping systems such as d irectional microphones, noise reduction schemes, time-frequency masking schemes are activated to reduce the cognitive load. The parameters in the helping systems are steered in accordance with the direct cognitive measure or the estimation from the cognitive model to reduce the cognitive load to a given residual cognitive spare capacity.

The term 'an estimate of present cognitive load' of a user is in the present context taken to mean an estimate of the present mental state of the user, the estimate at least being able to differentiate between two mental states HIGH and LOW use of mental resources (cognitive load). A LOW cognitive load is taken to imply a state of implicit processing of the current situation/information, which the user is exposed to (i.e. a routine situation, requiring no conscious mental activity). A HIGH cognitive load is taken to imply a state of explicit processing by the brain of the current situation/information, which the user is exposed to (i.e. a non-routine situation requiring mental activity). Acoustic situations requiring explicit processing of a user can e.g. be related to a bad signal to noise ratio (e.g. due to a noisy environment or a 'party'-situation) or to reverberation. In an embodiment, the estimate of present cognitive load comprises a number of load levels, e.g. 3 or 4 or 5 or more levels. In an embodiment, the estimate of present cognitive load is provided in real time, i.e. the estimate of present cognitive load is adapted to be responsive to changes in a user's cognitive load within seconds, e.g. in less than 10 s, e.g. less than 5 s, such as less than 1 s. In an embodiment, the estimate of present cognitive level is provided in as a result of a time-averaging process over a period, which is smaller than 5 minutes, such as smaller than 1 minute, such as smaller than 20 seconds.

In an embodiment, the method comprises providing a cognitive model of the human auditory system, the model providing a measure of the present cognitive load of the user based on inputs from customizable parameters, and providing said estimate of the present cognitive load of the user in dependence on said cognitive model.

In an embodiment, it is suggested to use an online individualized cognitive model in the hearing aid that determines when signal processing to reduce cognitive load should be used.

In an embodiment, the method comprises individualizing at least one of the customizable parameters of the cognitive model to a particular user's behavior.

One cognitive model that may be used is the Ease of Language Understanding model (Rόnnberg, 2003; Rόnnberg et al., 2008), which may predict when the cognitive load in a situation switch from implicit (effortless) to explicit (effortful). Thus the suggested use of the real-time ELU model would be to steer the aggressiveness of helping systems for the individual, in situations which are explicit/effortful for the individual. Other cognitive models may be used e.g. TRACE model (McClelland & Elman, 1986), the Cohort model (Marslen-Wilson, 1987) NAM model (Luce & Pisoni, 1998), the SOAR-model (Laird et al., 1987), the CLARION model (Sun, 2002; Sun, 2003; Sun et al., 2001 ; Sun et al., 2005; Sun et al., 2006), the CHREST model (Gobet et al., 2000; Gobet et al., 2001 ; Jones et al., in press) and the ACT-R model (Reder et al., 2000; Stewart et al., 2007), as well as Working Memory models according to Baddeley (Baddeley, 2000), however, according to the needs of the particular application.

In an embodiment, the processing of an input signal originating from the input sound according to a user's particular needs comprises providing a multitude of separate functional helping options, one or more of said separate functional options being selected and included in the processing according to an individualized scheme, depending on the input signal and/or on values of signal parameters derived there from, and on said estimate of the present cognitive load of the user.

In an embodiment, the separate functional helping options are selected from the group comprising (see e.g. Dillon, 2001 ; or Kates, 2008):

• directional information schemes,

• compression schemes

• speech detecting schemes

• noise reduction schemes • speech enhancement schemes,

• time-frequency masking scheme and combinations thereof.

This has the advantage that individual helping options can be taken into use or enhanced in dependence of an estimate of the cognitive load of a user, thereby increasing the comfort of the user and/or intell igibil ity of the processed sound.

In an embodiment, the properties or signal parameters extracted from the input signal include one or more of the following • amount of reverberation,

• amount of fluctuation in background sounds,

• energetic vs. informational masking,

• spatial information of sound sources

• signal to noise ratio, • richness of environmental variations and /or measures of auditory ecology (see e.g. Gatehouse et al. 2006 a,b). The latter properties or signal parameters dealing with 'richness of environmental variations' comprises e.g . short time variations in the acoustical environment as reflected in changes in properties or signal parameters of the input signal. In an embodiment, the parameters or properties of the input signal are measured with a number of sensors or derived from the signal. In an embodiment, acoustic dose is e.g. measured with a dose meter over a predefined time, e.g. seconds, e.g. 5 or 10 seconds or more (cf. e.g. Gatehouse et al., 2006 a,b; Gatehouse et al., 2003).

In an embodiment, the customizable parameters of the cognitive model relate to one or more of the following properties of the user

• Long-term memory capacity and access speed,

• Phonological awareness including explicit ability to manipulate the phonological units of words, syllables, rhymes and phonemes,

• Phonological working memory capacity,

• Executive functions: includes three major activities: shifting, updating and inhibition capacity (cf. e.g. Miyake & Shah, 1999),

• Attention performance (cf. e.g. Awh, Vogel & Oh, 2006), • Non-verbal working memory performance,

• Meaning extraction performance (cf. e.g. Hannon & Daneman, 2001 ),

• Phonological representations including phoneme discrimination, phoneme segmentation, and rhyme performance,

• Lexical access speed, • Explicit storage and processing capacity in working memory,

• Pure tone hearing thresholds vs. frequency,

• Temporal fine structure resolution (cf. e.g. Hopkins & Moore, 2007), and

• Individual peripheral properties of the hearing aid user including hearing thresholds and thresholds of uncomfortable listening, spectro-temporal and masking abnormalities in sensorineural hearing loss, (cf. e.g.

Gatehouse, 2006(a) and Gatehouse, 2006(b).

In an embodiment, the estimate of the present cognitive load of the user is determined or influenced by at least one direct measure of cognitive load for the user in question. In an embodiment, the estimate of the present cognitive load of the user is determined solely on the basis of at least one direct measure of cognitive load for the user in question. Alternatively, the estimate of the present cognitive load of the user is determined or influenced by a combination of inputs from a cognitive model and inputs from one or more direct measures of cognitive load of the user. In an embodiment, a direct measure of present cognitive load is used as an input to the cognitive model.

Any direct measure of current cognitive load can be used as an input to estimate current cognitive load. In a particular embodiment, however, a direct measure of cognitive load is obtained through ambulatory electroencephalogram (EEG).

In an embodiment, a direct measure of cognitive load is obtained through monitoring the body temperature.

In an embodiment, a direct measure of cognitive load is obtained through pupillometry.

In an embodiment, a direct measure of cognitive load is obtained through a push-button, which the hearing aid user presses when cognitive load is high.

In an embodiment, a direct measure of cognitive load is obtained in relation to a timing information, such as to the time of the day. Preferably, the timing information is related to a start time, such as the time the user awoke from a sleep or rest or the time when a user started on a work-related task (e.g. the stat time of a working period). In an embodiment, the method comprises the possibility for a user to set the start time.

A hearing aid system

A hearing aid system for processing an input sound and to provide an output stimulus according to a user's particular needs is furthermore provided by the present invention. The system comprises • an estimation unit for providing an estimate of present cognitive load of the user; • a signal processing unit for processing an input signal originating from the input sound according to the user's particular needs; • the system being adapted to influence said processing in dependence of the estimate the present cognitive load of the user.

In an embodiment, the hearing aid system comprises a hearing instrument adapted for being worn by a user at or in an ear. In an embodiment, the hearing instrument comprises at least one electric terminal specifically adapted for picking up electric signals from the user related to a direct measure of cognitive load. In an embodiment, the hearing instrument comprises a behind the ear (BTE) part adapted for being located behind an ear of the user, wherein at least one electric terminal is located in the BTE part. In an embodiment, the hearing instrument comprises an in the ear (ITE) part adapted for being located fully or partially in the ear canal of the user, wherein at least one electric terminal is located in the ITE part. In an embodiment, the system alternatively or additionally comprises one or more electric terminals or sensors NOT located in the hearing instrument but contributing to the direct measure of present cognitive load. In an embodiment, such additional sensors or electric terminals are adapted to be connected to the hearing instrument by a wired or wireless connection.

In an embodiment, the hearing instrument comprises an input transducer (e.g. a microphone) for converting an input sound to en electric input signal, a signal processing unit for processing the input signal according to a user's needs and providing a processed output signal and an output transducer (e.g. a receiver) for converting the processed output signal to an output sound. In an embodiment, the function of providing an estimate of the present cognitive load of the user is performed by the signal processing unit. In an embodiment, the functions of the cognitive model and/or the processing related to the direct measures of the cognitive load are performed by the signal processing unit. In an embodiment, the hearing instrument comprises a directional microphone system that can be controlled in accordance with the estimate of cognitive load. In an embodiment, the hearing instrument comprises a noise reduction system that can be controlled in accordance with the estimate of cognitive load. In an embodiment, the hearing instrument comprises a compression system that can be controlled in accordance with the estimate of cognitive load. The hearing instrument is a low power, portable device comprising its own energy source, typically a battery. The hearing instrument may in a preferred embodiment comprise a wireless interface adapted for allowing a wireless link to be established to another device, e.g. to a device picking up data related to direct measures of cognitive load of a user, e.g. voltages measured on body tissue of neural elements. In an embodiment, the estimate of present cognitive load of a user is fully or partially made in a physically separate device (from the hearing instrument, preferably in another body-worn device), and the result transmitted to the hearing instrument via a wired or wireless connection. In an embodiment, the hearing aid system comprises two hearing instruments of a binaural fitting. In an embodiment, the two hearing instruments are able to exchange data, e.g. via a wireless connection, e.g. via a third intermediate device. This has the advantage that signal related data can be better extracted (due to the spatial difference of the input signals picked up by the two hearing instruments) and that inputs to direct measures of cognitive load can be better picked up (by spatially distributed sensors and/or electric terminals).

It is intended that the process features of the method described above, in the detailed description of 'mode(s) for carrying out the invention' and in the claims can be combined with the system, when appropriately substituted by a corresponding structural features and vice versa. Embodiments of the system have the same advantages as the corresponding method.

A computer readable medium

A tangible computer-readable medium storing a computer program is moreover provided by the present invention, the computer program comprising program code means for causing a data processing system to perform the method described above, in the detailed description of 'mode(s) for carrying out the invention' and in the claims, when said computer program is executed on the data processing system.

A data processing system

A data processing system is moreover provided by the present invention, the data processing system comprising a processor and program code means for causing the processor to perform the method described above, in the detailed description of 'mode(s) for carrying out the invention' and in the claims.

Further objects of the invention are achieved by the embodiments defined in the dependent claims and in the detailed description of the invention.

As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well (i.e. to have the meaning "at least one"), unless expressly stated otherwise. It will be further understood that the terms "includes," "comprises," "including," and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements maybe present, unless expressly stated otherwise. Furthermore, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless expressly stated otherwise.

BRIEF DESCRIPTION OF DRAWINGS

The invention will be explained more fully below in connection with a preferred embodiment and with reference to the drawings in which:

FIG. 1 shows a hearing aid system according to a first embodiment of the invention,

FIG. 2 shows a hearing aid system according to a second embodiment of the invention, where cognitive model is used in the estimate of cognitive load, FIG. 3 shows a simplified sketch of the human cognitive system relating to auditory perception, and

FIG. 4 shows various embodiments of a hearing aid system according to the invention.

The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the invention, while other details are left out.

Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.

MODE(S) FOR CARRYING OUT THE INVENTION

FIG. 1 shows a hearing aid system according to a first embodiment of the invention.

The hearing instrument in the embodiment of FIG. 1 a comprises an input transducer (here a microphone) for converting an input sound (Sound-in) to en electric input signal, a signal processing unit (DSP) for processing the input signal according to a user's needs and providing a processed output signal and an output transducer (here a receiver) for converting the processed output signal to an output sound (Sound-out). In the embodiment of FIG. 1 (and FIG. 2), the input signal is converted from analogue to digital form by an analogue to digital converter unit (AD) and the processed output is converted from a digital to an analogue signal by a digital to an analogue converter (DA). Consequently, the signal processing unit (DSP) is a digital signal processing unit. In an embodiment, the digital signal processing unit [DSP) is adapted to process the frequency range of the input signal considered by the hearing instrument (e.g. between 20 Hz and 20 kHz) independently in a number of sub-frequency ranges or bands (e.g. between 2 and 64 bands or more, e.g. 128 bands). The hearing instrument further comprises an estimation unit (CL-estimator) for estimating the cognitive load of the user and providing an output CL indicative of the current cognitive load of the user, which is fed to the signal processing unit (DSP) and used in the selection of appropriate processing measures. The estimation unit receives one or more inputs (CL-inputs) relating to cognitive load and based thereon makes the estimation (embodied in estimation signal CL). The inputs to the estimation unit (CL-inputs) may originate from direct measures of cognitive load (cf. FIG. 1 b) and/or from a cognitive model of the human auditory system (cf. FIG. 2).

The estimation signal CL from the estimation unit is used to adapt the signal processing in dependence of CL (i.e. an estimate of present cognitive load).

FIG. 1 b shows an embodiment of a hearing aid according to the invention which differs from the embodiment of FIG. 1 a in that is comprises units for providing inputs to a direct measurement of current cognitive load of the user. In the embodiment of FIG. 1 b, measurement units providing direct measurements of current EEG (unit EEG), current body temperature (unit T) and a timing information (unit t). Embodiments of the hearing instrument may contain one or more of the measurement units or other measurement units indicative of current cognitive load of the user. A measurement unit may be located in a separate physical body than other parts of the hearing instrument, the two or more physically separate parts being in wired or wireless contact with each other. Inputs to the measurement units may e.g. be generated by measurement electrodes for picking up voltage changes of the body of the user, the electrodes being located in the hearing instrument(s) and/or in physically separate devices, cf. e.g. FIG. 4 and the corresponding discussion.

The direct measures of cognitive load can be obtained in different ways. In one embodiment, the direct measure of cognitive load is obtained through ambulatory electroencephalogram (EEG) as suggested by Lan et al. (2007) where an ambulatory cognitive state classification system is used to assess the subject's mental load based on EEG measurements (unit EEG in FIG. 1 b). See e.g. Wolpaw et al. (2002).

Such ambulatory EEG may be obtained in a hearing aid by manufacturing two or more for the purpose suitable electrodes in the surface of a hearing aid shell where it contacts the skin inside or outside the ear canal. One of the electrodes is the reference electrode. Furthermore, additional EEG channels may be obtained by using a second hearing aid (the other ear) and communicating the EEG signal by wireless transmission of the EEG signal to the other ear (e2e) or by some other transmission line (e.g. wireless through another wearable processing unit or through local networks, or by wire).

Alternatively, the EEG signal may also be input to a neural network to serve as training data with the acoustic parameters to obtain a trained network based on acoustic input and direct cognitive measures of cognitive load.

The EEG signal is of low voltage, about 5-100 μV. The signal needs high amplification to be in the range of typical AD conversion, (~2^"16 V to 1 V, 16 bit converter). High amplification can be achieved by using the analogue amplifiers on the same AD-converter, since the binary switch in the conversion utilises a high gain to make the transition from '0' to '1 ' as steep as possible. In an embodiment, the hearing instrument (e.g. the EEG-unit) comprises a correction-unit specifically adapted for attenuating or removing artefacts from the EEG-signal (e.g. related to the user's motion, to noise in the environment, irrelevant neural activities, etc.).

In another embodiment, direct measures of cognitive load can be obtained through monitoring the body temperature (unit T in FIG. 1 b), an increased/altered body temperature indicating an increase in cognitive load. The body temperature may e.g. be measured using one or more thermo elements, e.g. located where the hearing aid meets the skin surface. The relationship between cognitive load and body temperature is e.g. discussed in Wright et al. (2002). In another embodiment, direct measures of cognitive load can be obtained through pupillometry using eye-cameras. More contracted pupils mean relatively higher cognitive load than less contracted pupils. The relationship between cognitive (memory) load and pupillary response is e.g. discussed in Pascal et al. (2003).

In another embodiment, direct measures of cognitive load can be obtained through a push-button which the hearing aid user presses when cognitive load is high.

In another embodiment, direct measures of cognitive load can be obtained through measuring the time of the day, acknowledging that cognitive fatigue is more plausible at the end of the day (cf. unit t in FIG. 1 b).

FIG. 2 shows a hearing instrument according to a second embodiment of the invention, where cognitive model is used in the estimate of cognitive load.

The embodiment of a hearing instrument shown in FIG. 2 comprises the same elements as shown in FIG. 1 a and discussed in relation therewith. The hearing instrument of FIG. 2 further comprises a cognitive model of the human auditory system (CM in FIG. 2). The cognitive model (CM) is e.g. implemented as algorithms with input parameters received via input signals indicative of a users relevant mental skills (CM inputs in FIG. 2), typically customized to the user in question, and inputs indicative of relevant properties of the electric input signal (SP inputs in FIG. 2). Based on the inputs and the model algorithms one or more output signals (CL-inputs in FIG. 2) indicative of the cognitive load of the person in question is/are generated by the cognitive model (CM unit). These outputs are fed to the estimation unit (CL-estimator) for estimating the cognitive load of the user and providing an output CL indicative of the current cognitive load of the user, wh ich is fed to the signal processing unit (DSP) and used in the selection of appropriate processing measures. The output CL indicative of the current cognitive load of the user allows to at least differentiate between two mental states HIGH and LOW use of mental resources (cognitive load). Preferably more than two levels of estimated cognitive load are implemented, e.g. 3 levels (LOW, MEDIUM and HIGH). The cognitive model is e.g. implemented as part of a digital signal processing unit (e.g. integrated in the signal processing unit DSP in FIG. 2).

Based on the signal output(s) CL of the estimation unit, the signal processing unit (DSP) adapts its processing. The processing of the electrical input is a function of the cognitive load and characteristics of the input signal.

The user specific inputs (indicative of a user's relevant mental skills) to the cognitive model comprise one or more of parameters such as user age, user long term memory, user lexical access speed, user explicit storage and processing capacity in working memory, user hearing loss vs. frequency, etc. The user specific inputs are typically determined in advance in an 'off-line'- procedure, e.g. during fitting of the hearing instrument to the user.

The signal specific inputs to the cognitive model comprise one or more of parameters such as time constants, amount of reverberation, amount of fluctuation in background sounds, energetic vs. informational masking, spatial information of sound sources, signal to noise ratio, etc.

The appropriate processing measures taken in dependence of the inputs related to a user's cognitive load are e.g. selected among the following functional helping options, directional information schemes, compression schemes, speech detecting schemes, noise reduction schemes, time- frequency masking scheme, and combinations thereof.

The cognitive model (CM) shall, in real-time in the hearing instrument, predict to what extent at the moment explicit/effortful processing is required from the individual based on (a) parameters which may be extracted from the acoustical input (SP-inputs, e.g. amount of reverberation, amount of fluctuation in background sounds, energetic vs. informational masking, spatial information of sound sources) and (b) apriori knowledge of the individual persons' cognitive status (CM-inputs, e.g. WM capacity, spare resources, quality of long-term memory templates, speed of processing). In an embodiment, the hearing instrument is adapted to provide basis for online testing of the person's cognitive status. In an embodiment, the cognitive model is based on neural networks.

FIG. 3 shows a simplified sketch of the human cognitive system relating to auditory perception. An input sound (Input sound) comprising speech is processed by the human auditory system (Cognitive system, Perception). In an optimum listening situation, the speech signal is processed effortlessly and automatically (Implicit? YES => implicit processing). This means that the cognitive processing involved is largely unconscious and implicit. However, listening conditions are often suboptimum, which means that implicit cognitive processes may be insufficient to unlock the meaning in the speech stream (Implicit? NO => explicit processing). Resolving ambiguities among previous speech elements and constructing expectations of prospective exchanges in the dialogue are examples of the complex processes that may arise. These processes are effortful and conscious and thus involve explicit cognitive processing (Explicit). Both cases deliver some sort of perception of the input sound (Perception). The aim of the present invention is to include an estimate of current cognitive load (e.g. the differentiation between implicit and explicit processing of an incoming sound) in decisions concerning current optimum signal processing to provide an improved perception of the input sound for a user (compared to a situation where such decisions were taken based solely on the characteristics of the input sound signal and predefined settings of the hearing instrument, e.g. during fitting).

FIG. 4 shows various embodiments of a hearing aid system according to the invention. The hearing aid systems of FIG. 4 comprise a hearing instrument adapted for being worn by a user 1 at or in an ear. FIG. 4a shows an 'in the ear' (ITE) part 2 of a hearing instrument. In an embodiment, the ITE part constitutes the hearing instrument. The ITE part is adapted for being located fully or partially in the ear canal of the user 1. The ITE part 2 comprises two electric terminals 21 located on (or extending from) the surface of the ITE part. The ITE part comprises a mould adapted to a particular user's ear canal. The mould is typically made of a form stable plastic material by an injection moulding process or formed by a rapid prototyping process, e.g. a numerically controlled laser cutting process (see e.g. EP 1 295 509 and references therein). A major issue of an ITE part is that it makes a tight fit to the ear canal. Thus, electrical contacts on the surface (or extending from the surface) of the mould contacting the walls of the ear canal are inherently well suited for forming an electrical contact to the body. FIG. 4b shows another embodiment of a (part of a) hearing instrument according to the invention. FIG. 4b shows a BTE part 20 of a 'behind the ear' hearing instrument, where the BTE part is adapted for being located behind the ear (pinna, 12 in FIG. 4c and 4d) of a user 1. The BTE part comprises 4 electric terminals 21 , two of which are located on the face of the BTE part, which is adapted for being supported by the ridge where the ear (Pinna) is attached to the skull and two of which are located on the face of the BTE part adapted for being supported by the skull. The electric terminals are specifically adapted for picking up electric signals from the user related to a direct measure of cognitive load of the user. The electrical terminals may all serve the same purpose (e.g. measuring EEG) or different purposes (e.g. three for measuring EEG and one for measuring body temperature). Electrical terminals (electrodes) for forming good electrical contact to the human body are e.g. described in literature concerning EEG-measurements (cf. e.g. US 2002/028991 or US 6,574,513).

FIG. 4c shows an embodiment of a hearing aid system according to the invention, which additionally comprises an electric terminal 3 or sensor contributing to the direct measure of present cognitive load but NOT located in the hearing instrument 21. In the embodiment of FIG. 4c, the additional electric terminal 3 is adapted to be connected to the hearing instrument by a wired connection between the electric terminal 3 and one or both ITE parts 2. The electric terminal preferably comprises an electronic circuit for picking up a relatively low voltage (from the body) and for transmitting a value representative of the voltage to the signal processor of the hearing instrument (here located in the ITE-part). The wired connection may run along (or form part of the) flexible support members 31 adapted for holding the electric terminal in place on the head of the user. At least one of the additional electric terminals (here electric terminal 3) is/are preferably located in a symmetry plane of the head of the user (e.g. as defined by the line 1 1 of the nose of the user, the ears being located symmetrically about the plane) and e.g. constituting a reference terminal. FIG. 4d shows an embodiment of a hearing aid the system according to the invention, which additionally comprises a number of electric terminals or sensors contributing to the direct measure of present cognitive load, which are NOT located in the (here ITE) hearing instrument 2. The embodiment of FIG. 4d is identical to that of FIG. 4c apart from additionally comprising a body-mounted device 4 having 2 extra electric terminals 21 mounted in good electrical contact with body tissue. In an embodiment, the device 4 comprises amplification and processing circuitry to allow a processing of the signals picked up by the electric terminals. In that case the device 4 can act as a sensor and provide a processed input to the estimate of present cognitive load of the user (e.g. the estimate itself). The device 4 and at least one of the hearing instruments 2 each comprise a wireless interface (comprising corresponding transceivers and antennas) for establishing a wireless link 5 between the devices for use in the exchange of data between the body-mounted device 4 and the hearing instrument(s) 2. The wireless link may be based on near-field (capacitive of inductive coupling) or far-field (radiated fields) electromagnetic fields.

The invention is defined by the features of the independent claim(s). Preferred embodiments are defined in the dependent claims. Any reference numerals in the claims are intended to be non-limiting for their scope.

Some preferred embodiments have been shown in the foregoing, but it should be stressed that the invention is not limited to these, but may be embodied in other ways within the subject-matter defined in the following claims. REFERENCES

• Awh E., Vogel E. K. & Oh S. H. (2006), Interactions between attention and working memory, Neuroscience, Vol. 139, 2006, pp. 201-208. • Baddeley A. D. (2000). The episodic buffer: a new component of working memory? Trends Cogn. Science, Vol. 4, 2000, pp. 417-423.

• Dillon H. (2001 ), Hearing Aids, Thieme, New York-Stuttgart, 2001.

• Edwards B. (2007), The Future of Hearing Aid Technology, Trends in Amplification, Vol. 11 , No. 1 , 2007, pp. 31 -46. • Engle W., Kane J. M. & Tuholski S.W. (1999), Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In A. Myake & P. Shah (Eds.), Models of working memory, 1999, pp. 102-134, Cambridge (CUP). • EP 1 295 509 (PHONAK) 26-03-2003.

• Foo C, Rudner M., Rόnnberg J. & Lunner T. (2007), Recognition of speech in noise with new hearing instrument compression release settings requires explicit cognitive storage and processing capacity, J. Am. Acad. Audiol., Vol. 18, 2007, pp. 553-566. • Gatehouse, S., Naylor, G. & Elberling, C. (2003), Benefits from hearing aids in relation to the interaction between the user and the environment, Int. J. Audiol., Vol. 42 (Suppl 1 ), 2003, pp. S77-S85.

• Gatehouse et al. (2006a), Gatehouse S., Naylor G., Elberling C, Linear and nonlinear hearing aid fittings - 1. Patterns of benefit, Int. J. Audiol. Vol. 45(3), Mar. 2006, pp. 130-52.

• Gatehouse et al. (2006b), Gatehouse S., Naylor G., Elberling C, Linear and nonlinear hearing aid fittings - 2. Patterns of candidature, Int. J. Audiol., Vol. 45(3), Mar. 2006, pp. 153-71.

• Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C-H., Jones, G., Oliver, I. & Pine, J. M. (2001 ), Chunking mechanisms in human learning, TRENDS in Cognitive Sciences, Vol. 5, 2001 , pp. 236-243.

• Gobet, F. & Simon, H. A. (2000), Five seconds or sixty? Presentation time in expert memory, Cognitive Science, Vol. 24, 2000, pp. 651 -682.

• Hannon B. & Daneman M. (2001 ), A new tool for measuring and understanding individual differences in the component processes of reading comprehension, J. Educ. Psychol., Vol. 93 (1 ), 2001 , pp. 103- 128.

• R. C. Hendriks, R. Heusdens, and J. Jensen (2005), .Adaptive time segmentation of noisy speech for improved speech enhancement; in IEEE Int. Conf. Acoust., Speech, Signal Processing, March 2005, Vol. 1 , pp. 153.156.

• Hopkins & Moore (2007), Moderate cochlear hearing loss leads to a reduced ability to use temporal fine structure information, J Acoust. Soc. Am., Vol. 122(2), 2007, pp. 1055-1068. • Kates J. M. (2008), Digital Heaing Aids, ISBN13: 978-1 -59756-317-8: Plural Publishing Inc, San Diego, CA.

• Jones, G., Gobet, F., & Pine, J. M. (in press), Linking working memory and long-term memory: A computational model of the learning of new words. Developmental Science. • Lan et al. (2007), Lan T., Erdogmus D., Adami A., Mathan S. & Pavel M. (2007), Channel Selection and Feature Projection for Cognitive Load Estimation Using Ambulatory EEG, Computational Intelligence and Neuroscience, Volume 2007, Article ID 74895, 12 pages.

• Li K.Z.H., Lindenberger U., Freund A.M. & Baltes P. B. (2001 ), Walking while memorizing: Age-Related differences in compensatory behavior.

Psychol. Sci., Vol. 12 (3), 2001 , pp. 230-237.

• Laird, Rosenbloom, Newell, John and Paul, Allen (1987). "Soar: An Architecture for General Intelligence". Artificial Intelligence, Vol. 33, 1987, pp. 1 -64. • Luce P.A. & Pisoni D. A. (1998), Recognising spoken words: the neighbourhood activation model, Ear Hear, Vol. 19, 1998, pp. 1 -36.

• Lunner T. (2003), Cognitive function in relation to hearing aid use, Int. J. Audiol., Vol. 42 (Suppl. 1 ), 2003, pp. S49-S58.

• Lunner T. & Sundewall-Thoren E. (2007), Interactions between cognition, compression , and listening conditions: effects on speech-in-noise performance in a two-channel hearing aid, J. Am. Acad. Audiol., Vol. 18, 2007, pp. 539-552.

• Marslen-Wilson W. 1987. Functional parallelism in spoken word recognition. Cognition, Vol. 25, 1987, pp. 71 -103. • McClelland, J. L. & Elman, J. L. 1986. The TRACE model of speech perception. Cogn Psychol, Vol. 18, 1986, pp. 695-698. • Miyake A. & Shah P. (1999), Models of Working Memory, Cambridge, UK, Cambridge University Press.

• Naylor G., Johannesson R. B. & Lunner T. (2006), Fast-acting compressors change the effective signal-to-noise ratio - both upwards and downwards, International Hearing Aid Research Conference

(IHCON), Lake Tahoe, CA, August 2006.

• Pascal W. M. Van Gerven, Fred Paas, Jeroen J. G. Van Merrienboer, and Henrik G. Schmidt, Memory load and the cognitive pupillary response in aging, Psychophysiology. Volume 41 , Issue 2, Published Online: 17 Dec 2003, Pages 167 - 174.

• Reder, L. M., Nhouyvanisvong, A., Schunn, C. D., Avers, M. S., Angstadt, P., & Hiraki, K. (2000). A mechnistic account of the mirror effect for word frequency: A computational model of remember-know judgments in a continuous recognition paradigm. Journal of Experimental Psychology: Learning, Memory, and Cognition, Vol. 26, 2000, pp. 294-320.

• Rόnnberg J. (2003). Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: A framework and a model, Int. J. Audiol., Vol. 42, 2003, pp. S68-S76.

• Rόnnberg, J., Rudner, M., Foo, C. & Lunner, T. (2008), Cognition counts: A working memory system for ease of language understanding (ELU),

International Journal of Audiology, Vol. 47 (Suppl. 2), 2008, pp. S171 - S177.

• Sarampalis A., Kalluri S., Edwards B., Hafter E. (2008), Understanding speech in noise with hearing loss: measures of effort, International Hearing Aid Research Conference (IHCON), Lake Tahoe, CA, August 13-

17, 2008.

• Stewart, T.C. and West, R. L. (2005) Python ACT-R: A New Implementation and a New Syntax. 12th Annual ACT-R Workshop

• Sun, R. (2002). Duality of the Mind: A Bottom-up Approach Toward Cognition. Mahwah, NJ: Lawrence Erlbaum Associates.

• Sun, R. (2003). A Tutorial on CLARION 5.0. Technical Report, Cognitive Science Department, Rensselaer Polytechnic Institute.

• Sun, R., Merrill, E., & Peterson, T. (2001 ). From implicit skills to explicit knowledge: A bottom-up model of skill learning. Cognitive Science, 25, 203-244. http://www.cogsci.rpi.edu/~rsun/ • Sun, R., Slusarz, P., & Terry, C. (2005). The interaction of the explicit and the implicit in skill learning: A dual-process approach. Psychological Review, Vol. 112, 2005, pp. 159-192. http://www.cogsci.rpi.edu/~rsun/

• Sun, R. & Zhang, X. (2006). Accounting for a variety of reasoning data within a cognitive architecture. Journal of Experimental and Theoretical

Artificial Intelligence, Vol. 18, 2006, pp. 169-191.

• Retrieved from "http://en.wikipedia.org/wiki/CLARION_(cognitive_architecture)"

• US 2002/028991 (MEDTRONIC) 07-03-2002 • US 6,574,513 (BRAINMASTER TECHNOLOGIES) 03-06-2003

• Wang D. L., Kjems U., Pedersen M.S., Boldt J. B. & Lunner T. (Conditionally accepted). Speech Intelligibility in Background Noise with Ideal Binary Time-frequency Masking, Journal of the Acoustical Society of America. • Wolpaw J. R., Birbaumer N., McFarland D.J., Pfurtscheller G. & Vaughan T. M. (2002), Brain-computer interfaces for communication and control, Clinical Neurophysiology, Vol. 113, 2002, pp. 767-791.

• Kenneth P. Wright Jr., Joseph T. Hull, and Charles A. Czeisler (2002), Relationship between alertness, performance, and body temperature in humans, Am. J. Physiol. Regul. Integr. Comp. Physiol., Vol. 283, August

15, 2002, pp. R1370-R1377.

Claims

1. A method of operating a hearing instrument for processing an input sound and to provide an output stimulus according to a user's particular needs, comprising a) providing an estimate of the present cognitive load of the user; b) providing processing of an input signal originating from the input sound according to a user's particular needs, c) adapting the processing in dependence of the estimate the present cognitive load of the user.

2. A method according to claim 1 comprising a1 ) providing a cognitive model of the human auditory system, the model providing a measure of the present cognitive load of the user based on inputs from customizable parameters, and a2) providing said estimate of the present cognitive load of the user in dependence on said cognitive model.

3. A method according to claim 2 comprising a1.1 )individualizing at least one of the customizable parameters of the cognitive model to a particular user's behavior.

4. A method according to any one of claims 1 -3 wherein said processing of an input signal originating from the input sound according to a user's particular needs comprises b1 ) providing a multitude of separate functional helping options, one or more of said separate functional options being selected and included in the processing according to an individualized scheme, depending on the input signal and/or on values of signal parameters derived there from, and on said estimate of the present cognitive load of the user.

5. A method according to claim 4 wherein the separate functional helping options are selected from the group comprising • directional information schemes, • compression schemes, • speech detecting schemes,

• speech enhancement schemes,

• noise reduction schemes,

• time-frequency masking schemes and combinations thereof.

6. A method according to claim 4 or 5 wherein signal parameters extracted from the input signal include one or more of the following

• amount of reverberation, • amount of fluctuation in background sounds,

• energetic vs. informational masking,

• spatial information of sound sources,

• signal to noise ratio,

• richness of environmental variations.

7. A method according to any one of claims 2-6 wherein customizable parameters of the cognitive model relate to one or more of the following properties of the user

• long-term memory capacity and access speed, • phonological awareness including explicit ability to manipulate the phonological units of words, syllables, rhymes and phonemes,

• phonological working memory capacity,

• Executive functions: includes three major activities: shifting, updating and inhibition capacity, • Attention performance,

• Non-verbal working memory performance,

• Meaning extraction performance,

• Phonological representations including phoneme discrimination, phoneme segmentation, and rhyme performance, • lexical access speed,

• explicit storage and processing capacity in working memory

• pure tone hearing thresholds vs. frequency,

• temporal fine structure resolution, and

• Individual peripheral properties of the hearing aid user.

8. A method according to any one of claims 1 -7 wherein said estimate of the present cognitive load of the user is determined or influenced by at least one direct measure of cognitive load for the user in question.

9. A method according to claim 8 wherein a direct measure of cognitive load is obtained through ambulatory electroencephalogram (EEG).

10. A method according to claim 8 or 9 wherein a direct measure of cognitive load is obtained through monitoring the body temperature.

11. A method according to claim any one of claims 8-10 wherein a direct measure of cognitive load is obtained through pupillometry.

12. A method according to claim any one of claims 8-11 wherein a direct measure of cognitive load is obtained through a push-button, which the hearing aid user presses when cognitive load is high.

13. A method according to claim any one of claims 8-12 wherein a direct measure of cognitive load is obtained in relation to a timing information, such as the time of the day.

14. A hearing aid system for processing an input sound and to provide an output stimulus according to a user's particular needs, the hearing aid system comprising

• an estimation unit for providing an estimate of present cognitive load of the user;

• a signal processing unit for processing an input signal originating from the input sound according to the user's particular needs; the system being adapted to influence said processing in dependence of the estimate the present cognitive load of the user.

15. A hearing aid system according to claim 14 comprising a hearing instrument adapted for being worn by a user at or in an ear, the hearing instrument comprising at least one electric terminal specifically adapted for picking up electric signals from the user related to a direct measure of cognitive load.

16. A hearing aid system according to claim 14 or 15 comprising one or more electric terminals or sensors NOT located in the hearing instrument but contributing to the direct measure of present cognitive load.

17. A tangible computer-readable medium storing a computer program, comprising program code means for causing a data processing system to perform the method of any one of claims 1 -13, when said computer program is executed on the data processing system.

18. A data processing system, comprising a processor and program code means for causing the processor to perform the method of any one of claims 1 -13.