Adv Sig Prcssing Part 2 mp4

Advanced Signal Processing Part 2 Lecture 3/25/20

Dr. Jodi Baxter >>Okay, so what is the purpose of directional microphones or directional technology?

So directional technology was developed in the 20s and 30s and what I remember of this -

- unfortunately I did not write it down -

-was that it was actually initially developed for places like stadiums,

for football games, basketball games in order to have audibility

for what was going on on the court or on the field,

and not have what was happening in the crowd

or the stadium interfering as much with that sound.

The first hearing aid that was directional came out from Oticon in 1972 for the -

- that hearing aid in the 80s,

it was that you were purchasing a directional hearing aid.

And so that was implemented in the way of having that single microphone with two ports,

which we had talked about back in the hearing aid components lecture.

So it was a very small number of hearing aids

that were sold that were directional hearing

aids and not really stayed that way and even declined throughout the 80s.

So there wasn't a lot of availability or options in analog hearing aids,

and it wasn't until the early 90s that the option became available to switch

manually from an omnidirectional hearing aid or hearing aid mode program to a directional mode.

And this was again still pretty limited in terms

of what the size of that directional array was,

and it was fixed and it was using,

again, a manual toggle between the two modes.

So Phonak actually was the first manufacturer to come out with the

concept of a twin microphone or having two omnidirectional microphones and

since then, we've been really able to run with that technology.

So I would like to encourage you to take a couple of minutes to go back

to the hearing aid components lecture and review

the two types of directional microphone technology that we discussed,

which was the single microphone that had two different ports,

which I believe is not used at this point anymore.

And the multi microphone system,

meaning that there are multiple,

typically two per hearing aid omnidirectional microphones.

And then now, because we have this binaural communication,

they're really calling it a microphone array or a dual

microphone, so meaning the communication between those microphones

provide the hearing aid with information on where

it should be having maximal sensitivity to sound.

So when we're talking about a polar plot or a polar sensitivity pattern,

we're really looking at the three-dimensional -

- dimensional space surrounding the

microphone and where it is most sensitive to sound.

Again, if you take a few minutes to go back to the hearing aid components lecture and review that,

that's really kind of determined by those two

design parameters that we initially talked about.

So the external delay, meaning the port spacing between the two microphones,

which is typically between 4 and 12 millimeters,

and then that internal delay which,

at this point, is typically done mathematically or electrically.

So a polar plot is a plot of the hearing aid output

as a function of the angle of the sound source in the horizontal plane.

So typically the way polar plots are developed

is by having a keymar or an acoustic mannequin in the center of a speaker array,

a 360-degree array of speakers,

and then plotting the output of the hearing aid,

depending on where the signal of interest or sound source is,

what speaker it's coming from.

It's also often referred to as the directional sensitivity pattern.

A vocabulary word you need to know is the null

point which is the point of greatest attenuation or the least amount of sensitivity.

So if we look at figure 1027 here,

we're looking at - - this kind of center area is where the acoustic mannequin is,

and then this is the speaker array,

and you can see the dotted is the omnidirectional BTE.

Remember we talked about, it's really not truly omnidirectional if it's

mounted on someone's head because of the head shadow effect.

So my guess by looking at this is that the hearing aid is a mounted on the right ear,

because we have greatest sensitivity here and a little bit less on this side.

And then we have the directional sensitivity pattern here,

so greatest sensitivity in this area,

and then a null point here and less sensitivity to this side of the head.

And then what's important to also understand is that this varies by frequency as well.

So remember that directional microphones operate by sampling at multiple locations,

and comparing the sound sample at those locations.

So remember the external delay is simply the spacing between the two microphones,

or the microphone ports. And calculating that spacing or calculating the external delay by dividing the spacing itself.

So the physical distance between the speed of sound.

The internal delay, back in these original directional microphones,

where there was one microphone,

two ports- - a damper was put into the port of the rear microphone or the back microphone.

So this was typically like a wire mesh acoustic damper.

And the purpose was to introduce a time delay

by slowing down the sound as it arrived to the rear microphone.

So if you think about it, sound coming from behind the hearing aid

user would arrive to this microphone first and then travel to the front microphone.

By introducing the damper here and flowing down that sound,

the goal is that the sound that arrives to the

rear port and goes to the underside or bottom of the diaphragm,

would arrive at the same time as the sound arriving to the front microphone,

which is sent to the top or the upper side of the diaphragm.

And if they are arriving at the same time,

then there's essentially no net pressure on the diaphragm and that sound is canceling.

It's canceled out. So the more exactly matched the phase is,

the greater the attenuation is.

Now that manipulation or that relationship between the internal delay

and the external delay is primarily done mathematically or electronically.

And that is what determines what a polar plot looks like.

So again, omnidirectional means that the microphone is equally sensitive to

sounds from all direction, all directions.

Again when we're looking at one that's a perfect circle like this,

it's suspended in free space,

so it's not taking into account the hearing aid user wearing the

hearing aid. So again, our head shadow effect.

This is good when there - - it's a

quiet environment, because it gives the patient access to all sounds in the environment.

And then this is an example of a directionality pattern,

a cardioid pattern is what it's called because it looks like a little heart.

When you're looking at this type of polar plot,

the hearing aid microphone is sensitive to sounds in front of the hearing aid user,

so that's gonna be the zero degree here,

and there's a large null point or a point where it's not as sensitive to

sounds behind the hearing aid user.

The other thing that's important to

understand and we'll see here in a minute is that these polar plots are

also frequency-dependent. So this is a series of other versions of cardioid -

- or polar plot - - you can ignore this one.

I actually pulled this from something that was not hearing aid

related just so that - - I thought it was the best image I could find.

So we also have a - - let's see,

super cardioid here, so again more sensitive to sounds behind the user than a plain cardioid,

but narrow sensitivity to sounds in front compared to a regular cardioid.

Same thing with a hyper-cardioid,

so even more narrower or more directional in the front,

as you can see - - although it's not as obvious.

But again, greater sensitivity to sounds in the rear and these kind of null points

on the sides. Figure 8, or a bi-directional is equally as sensitive to the front and back,

but not as sensitive to sounds to the side.

Yeah, so just options of what sensitivity and null points are.

Okay, so directivity index is -

- (break in audio) - - - What makes this a little bit less straightforward

and a little bit more confusing is at this point,

directionality in all digital systems are adaptive;

they switch automatically and they're multiband.

So each frequency is often doing something different.

It's not as global as it used to be.

So, but some more terminology that's useful to understand.

Fixed directionality is when the sensitivity pattern does not adapt or does not change.

Fixed directionality does not actually have to be what I have written

on this slide here which is one null point behind the patient or a cardioid.

Fixed just really means that it's not adaptive and at this point,

we do have the ability to set up fixed directionality in most hearing aids as a

separate program, so times that I may choose to do this is if,

for example, a patient is still continuing to struggle,

and it doesn't seem like the adaptive

algorithms of the hearing aid are as sensitive as the person needs it to be

or they're describing something of it kind of coming in and out of directionality,

I could put a fixed kind of tight directional channel so a beam former,

which we'll talk about here in a minute,

directly in front of them, so that they

can manually override those decision rules that are in the hearing aid

programming and the patient can use their phone app or their program button

on the hearing aid to go into that directional program.

That tends to be a little bit more effective

in terms of providing a better signal-to-noise ratio,

but there are some challenges too;

for example, the patient has to know how to use it,

so how to actually change into it,

but they also have to remember when to change into it.

They have to physically be able to do that;

again either the push button on the hearing aid,

or through their app. And they also have to remember to get out of it which,

anecdotally, is often a challenge of,

okay, now I'm leaving this restaurant and I chose to

put my hearing aid into the directional program and now I'm leaving and I

completely forgot to move it out of that.

Which then, say those two individuals get in the car,

they're really going to struggle if they're in a fixed directional program and the array is

focused on sounds directly in front of them.

If they're driving, they're not going to hear the person to the right of them in the car at all.

So it's important that if you're gonna set up a fixed program,

you've got to choose the patient wisely,

and you really have to counsel on when to use it and when not to use it.

Another challenge again of that is,

say, two people are at a restaurant and then the server comes up and starts talking,

the person in that fixed program -

- again, of course, depending on their hearing loss and other features in the hearing aid,

they may not have any awareness that the server has come up and started talking,

because they have a null point in that spot,

you know, to the side and really only has good sensitivity

or ability to pick up sounds from directly in front of them.

Other scenarios where I've set up fixed directional programs,

if you think of someone in a wheelchair or someone who's driving children,

maybe, you know, parent bus driver -

- I actually have set up a fixed directional program for a bus driver.

Think about it. Where that person needs to hear is actually

not in front of them but sounds behind them.

So we often have the ability to make a fixed directivity pattern,

you know, in front, behind to the side,

depending on what the need of your patient is,

but that - you need to have, you know,

lengthy conversations and counseling sessions

on what their need is and how to use it effectively;

otherwise they could potentially get into trouble.

In a fixed sensitivity pattern,

that time delay is constant. In an adaptive pattern,

that directivity pattern typically is changing

to maximize the audibility from a target direction or more often,

the most dominant sound source.

So, in general, these are usually more effective than the fixed arrays,

because it's rare that an individual only wants to hear from

one direction. So in this, that time delay is constantly varied,

which then changes the polar plot constantly to provide the most attenuation for,

you know, depending on what the sound level is to the side,

from behind. Again, there are typically,

just like what we talked about in the last lecture,

digital - -the digital noise reduction lecture -

-there are usually rules or criterion in

place for the hearing aid to decide,

okay, at this point, so for example,

when sound exceeds 55 or 65 DB SPL,

which I would say is typically really conservative or more often,

it's like a 70 DB SPL and the SNR is of a certain range in this

frequency band - - the hearing aid will go into a directionality mode.

But when that criterion has not been met,

the hearing aid is functioning omnidirectionally.

Again, this is multi-channel,

so this is happening at multiple frequency channels at any given time.

So in- - within any frequency band,

the hearing aid is looking for a noise source,

detecting this overall DB SPL,

detecting the signal-to-noise ratio,

and then applying those different time delays in each channel.

And so the goal is to reduce multiple noise sources that are spatially separated,

if they're occurring in different frequency regions.

Okay, so beamforming directionality.

This really came to be when we had the

development of that, again, intraoral or intra ear wireless communication

because that essentially allowed us to then have four microphones because,

again, we have two on each hearing aid that communicate with one another.

So this allowed us to move towards even more narrow directionality.

When we're talking about beamforming,

though, it's exactly that. So you can think of it as a beam and I really like this graphic here,

where the purple is what is within that beam or that very narrow directionality,

that is getting good emphasis or good sensitivity and everything outside of

that beam is attenuated. So a traditional dual mic system

has about a sixty degree angle,

whereas beamforming technology can be as

tight as about 45 degrees. The limitations,

I've kind of already talked about so,

for example, if you're using

this beam in a restaurant setting and then the server comes up to the side,

they're not going to have probably even awareness -

- again, of course, depending on degree of hearing loss in the speaker and a number of other things,

but much less awareness of sound coming from other directions.

So I have some people, for example,

who may use this in church, but again,

they're not gonna be able to communicate at that point with the

person sitting next to them.

So counseling is really, really important

when you are choosing to use this type of directionality.

And you can see here, I'm pretty sure this is Signia.

I do like this app, although I don't use that manufacturer

as often, but this is a really nice way for the patient to visually understand and

control what they - - where their directional beams are pointing to and then you can see down here at the bottom,

they also have the ability to control how wide or narrow that beam is.

In general, the smaller that angle is,

the more favorable the signal-to-noise ratio is.

So evidence has shown that when we,

in the lab setting, the angle goes from 60 degrees,

which is a traditional directional mic system to about 40 degrees -

- 45 degrees - - that can make an improvement as great as 1 DB in the signal-to-noise ratio.

So there are a number of manufacturers

that are implementing this beamforming technology,

although not all of them. What you're looking at here on these polar plots -

- these were Signia, although not the latest platform now,

and we're looking at, you know,

them bragging about their beamforming technology

in their hearing aids versus other manufacturers,

but mostly what - - I just liked the ability to look at how those polar plots may vary,

again, for different manufacturers,

different technology. Some manufacturers use split band or split channel directionality.

So what this is, is the hearing aid is directional in the

high frequencies, but omnidirectional and the low frequencies.

This cutoff point of where it becomes directional

versus omnidirectional may be fixed by the manufacturer;

it may be something that's able to be adjusted by the clinician or audiologist.

Or it may vary depending on the patient's hearing loss.

Typically that cutoff point is around two thousand Hz or so.

The purpose of this is that direction -

- directional systems typically have a low frequency roll-off naturally.

That is then compensated for by applying an extra

boost of amplification in the low frequencies.

This is called equalization.

But when that low frequency boost is provided,

it then affects the effectiveness of the directionality of the system.

At the same time, if you don't do it,

the patient may complain that the hearing aid sounds too tinny or too unnatural.

The advantages of having this concept of directional

in the higher frequencies and omnidirectional in the low frequencies

is that it does more closely mimic the directivity of the human ear.

So we take advantage of the directionality,

but also we preserve the environmental awareness and low frequency cues that happen,

such as the interaural timing difference.

If the equalization concept -

- so again, that idea that the low frequencies are boosted

in order to compensate for that low frequency

roll-off that happens naturally with directional systems,

then the individual's own voice,

wind noise, other low frequency sounds,

may be overamplified. But this is gonna sound more natural to the hearing aid user.

The disadvantage of having this concept of split

band or split channel directionality is that

there will be no signal-to-noise ratio improvement in the

low frequencies, which is where noise often occurs.

So sometimes that may not be great for hearing and noise.

The other concept to keep in mind is that all

open fit hearing aids really are split directionality.

And the reason for that is, remember in the low frequencies,

if you have something that's truly open,

like open dome, we're really not providing

gain in those frequencies at all and the sound is able to enter the ear canal

naturally, the way it normally does.

A hearing aid isn't going to be acting on

any sound that is entering the ear canal naturally.

So if it's not being processed by the hearing aid,

then it's not getting any directionality.

There have definitely been times working

with patients where I may move to more occluding or closed canal,

by way of dome or earmold sooner than I thought I would based on their hearing loss.

So they may have, you know, decent low frequency hearing,

but I may choose to go with something a little bit more occluding because I really need

that directionality, noise reduction,

any of that to take place in the low frequencies.

This is not common but it's something that I definitely think about

and have to balance as I'm making decisions.

Again, this is typically not the way I'll go right off the bat,

because most of the time, if you include the ear more than it should be,

the patient is not going to be satisfied with the sound quality or the

naturalness of their own voice,

particularly, because of the occlusion effect.

But if I'm continuing to have a patient struggle or complain about hearing and noise or,

you know, things that make it sound like the noise reduction

aren't doing their job, and the patient is in a completely open dome,

that may be something that I consider changing.

So again, we're looking at polar plots here.

But what we're looking at now is the polar plot by frequency.

So you can see there's different colors to demarcate the different frequencies,

so 500, 1000, 2000, 4000. And this graph is really meant to show you

both in a completely open ear and also with a

hearing aid that is intentionally doing split band directionality,

the low frequencies, so 500 and 1000 hz are essentially omnidirectional versus two and four,

which are getting some level of directionality.

So again, these, you know, anything that falls outside of this red

or this gold region for the two and four K,

the hearing aid is really not picking up.

It's not sensitive to it, so you can see the difference in sensitivity patterns

between the microphone between those different frequencies.

One manufacturer, G.N. Resound,

uses asymmetric directivity and to be completely honest with you,

the first time I heard about this type of directionality,

and even now, as I read about it,

it's hard for me to understand that it's effective.

But the evidence shows that it is.

What this is is, one hearing aid is always in

omnidirectional and one hearing aid is always in directional.

The theory behind this or the goal,

is to maximize auditory awareness from any direction.

I say that in quotes because we still have

to remember that we have the concept of the head shadow coming into play here.

But also having directionality at all times as well.

They reported that this was developed initially to overcome a patient's inability to use the manual

directional microphone, and the limitations that do exist of automatic

environmental classification.

The last I heard, this was still being used

in Resound hearing aids, but hopefully the group that has been assigned to look

at Resound can tell us more about it.

Again, there is research to show that

it's successful, so two different articles showed just as much directional

benefit - - and then two articles that showed less directional benefit.

So 1-1/2 to 2 dB in adults and children with mild to moderate hearing loss.

Okay so let's talk a little bit about reverberation.

Reverberation is when a sound source is reflected

off room surfaces and so in this scenario we

have a direct sound pathway,

and multiple reflected sound pathways.

As the distance from the speaker,

or the sound source increases,

the direct sound level will decrease,

but the reverberant sound remains constant depending on the

location of the reflective services.

So, for example, if you are near a wall or a

hard surface that may be a reflective surface,

the reflective sound or the reverberant sound may even be higher than the direct sound.

Critical distance is an important concept to understand.

Critical distance is the point that the direct sound is equal to the reverberant sound.

So basically once you're outside of that critical distance,

the direct sound level is drastically reduced compared to the reverberant sound.

And once we're outside of that critical distance point,

the directional microphone and the digital noise

reduction effectiveness are also reduced.

So this is why counseling is very important,

even with these great features

that we have of directional microphones and digital noise reduction,

the patient still has to be close to the speaker.

This will improve the signal-to-noise

ratio, this reduces the consequential effects of reverberation.

This reduces the effects of distance,

so think about your inverse square law.

The further you move away from the sound,

the greater the reduction of sound,

and this also does allow the directional microphones to be more effective.

Finally, it gives the patient access to visual cues from the speaker.

Typically, critical distance is around six feet.

Some of the manufacturers say that directionality really is most effective within 10 feet.

So again, this is important for you to remember as the clinician.

A perfect example of this is church.

You know, even if the patient is in the very front row,

they are rarely within 10 feet of the person talking.

Now, again, this is a different type of environment because they do have speakers

too - - but you can set up the most beautiful fixed directional program,

but if the speaker is still 30 feet away,

it's not going to be as effective.

So this is why we need to have a constant conversation about,

again, being close to the speaker for a variety of different reasons.

Reverberation is a pretty significant challenge:

the biggest challenge for directional microphones.

If we look at our waveform graphs or figures on the

right side of the slide here,

what you can see happens is reverberation

essentially fills in the modulation depth of the speech signal.

So this is the same signal, 4.8 and 415,

but this signal is sampled in a reverberant room.

So you can see - - we have these really big peaks and valleys and

modulation depths here that are essentially wiped away or gone in Figure 415.

This isn't noise; this is just a reverberation.

Average reverberation time is 4 seconds.

The longer the time is, the more that modulation is filled in.

So basically what's happening here is earlier parts of speech that are then

reverberating are serving to mask part -

- later occurring parts of speech.

Another challenge with reverberation in terms of directional microphones is that

essentially it's reaching the microphone evenly from all directions.

So then those concepts of making a polar plot and

putting in a null point really works minimally well when the noise is coming

from every direction. So again,

an adaptive algorithm tries to choose -

- choose the pattern that is average from overall directions to attenuate the most noise.

But that's going to be challenging when the noise is really coming from everywhere.

So clinical application of directional microphones.

At this point they're primarily implemented automatically and adaptively,

but there are scenarios, as I

described earlier in the lecture that they can be implemented as a manual program.

You just have to make sure the patient knows how and when to

successfully use the program.

The mic port location orientation have to be

correct and in the horizontal plane in order for them to be effective.

The range of benefit of directional microphones is really highly dependent on the environment.

So hopefully we've covered this in depth throughout this slide show,

but factors, of course, are going to include the number,

location and type of competing noise sources,

reverberation time in the environment,

the distance between the hearing aid user and sound of interest,

and the fact that the hearing aids are operating on the principle that the speech is the signal

of interest. And typically operating on the principle that the loudest speech is

the clinical - - the signal of interest,

which may not always be the case.

Another thing to keep in mind is the effective venting on directionality,

which we talked about briefly a couple of slides ago.

So the hearing aids style doesn't -

-has not been found in the research to have a significant impact on directionality.

But venting or openness of the canal does.

Again, because sound - - sound is leaking out of the ear canal,

but also because sound is entering the ear canal unprocessed by the hearing aid.

So if you look at figure 1116 here,

we're looking at changes in low frequency DI,

or directivity index. In AIDI which is the articulation index directivity

index for a no-vent versus a 1-millimeter event,

which is quite small, so that's gonna be a little bit larger than a pressure vent.

A 2-millimeter vent, which is,

then, a pretty large vent and then a completely open canal.

But, you know, venting is valuable and omnidirectional is valuable,

in that it does provide us with sound awareness from multiple directions.

So what does the evidence tell us on directionality

and speech recognition and noise?

This is wide-ranging. It's been reported that up to 11 DB improvement

or 70 percent improvement in word recognition in noise in a lab setting,

when using directionality. A 2017 article by Todd Ricketts at Vanderbilt

reported that directional microphones provide

an advantage or improvement in signal-to-noise ratio about 42 percent of the time in school.

This is important because for a number of years,

it was said that we shouldn't be using directionality

or noise reduction in pediatric fittings because we were

potentially not able to take advantage of the concept of incidental learning.

At this point it seems like enough evidence has

disproven that directional microphone noise reduction would be valuable for kids in schools,

but we will talk more about that in Hearing Aids II when we have a pediatric-specific lecture.

Adaptive directionality and beamforming directionality

have been shown to add an additional 1 to 2 DB

of benefit over that concept of traditional fixed directionality.

Listening effort - - uh, so I'd love to see the research coming out about listening effort.

Both directional and beamforming technology have been shown to

reduce listening effort, both behaviorally and subjectively compared

to omnidirectional implementation.

So, again, this is where we have to think

about how we use this for patients.

This study Chord et al. , found that 30% of

users did not switch between settings and users often did not know when to switch.

Anecdotally I will tell you that for the most part that seems to be true.

I really try to avoid setting up specific programs for patients because

even the most savvy patients seem to stop using them after a period of time,

or still don't quite understand how to use them,

when to use them, when to get out of them.

I do do them, sometimes that you just absolutely have to,

but I would be conservative with doing that and making sure you're counseling and also

checking back in routinely because I troubleshooted a lot of problems

because of excessive program use and lack of understanding of programs.

One study with Terese Walden suggested that some

of the switching algorithms may be too conservative,

so it's - - the study suggested that 33% of the time,

hearing users were in environments where a directional benefit may be -

- the directional microphone may be beneficial;

however, they only switch 5 to 17 percent of the time.

I will tell you, also anecdotally,

this has been my experience clinically;

in the manufacturers that do have really precise data logging systems,

I am shocked by how rarely they're actually going

into their speech in noise program or programs

where directionality and noise reduction should be -

- are used, so that information can be valuable,

especially when you have the ability to change how quickly the hearing aid is doing this.

So again, looking at directional advantage,

compared to omni, it's a wide range,

and replicated by a number of studies.

There does not seem to be a significant effect

based on hearing loss in terms of omnidirectional versus directional.