Module 10.2. 
Some research projects that use the

techniques you have learned in the 
digital signal processing class.

We're going to talk about some current 
research in the lab.

There is a whole slew of them, and it's a 
selection here of interesting research

projects that we can briefly discuss. 
The first one is, eFacsimile is a project

on art work acquisition. 
The second one is about signal processing

in sensor networks. 
Then there is a network science result on

source localization put in graphs. 
Then we talk about sampling result, so

called finite rate of innovation 
sampling.

Then we talk again about sampling, that 
of physical fields, using some new

techniques for sampling. 
Then, we have a project on image

acquisition where we change the sensors 
used in acquiring images.

Then, an old classic, predicting the 
stock market.

Then, we talk about inverse problems. 
The three next projects are actually

inverse problems. 
The first one is on the diffusion

equation, the second one is trying to 
understand the nuclear fall out from

Fukushima, and last but not least is an 
inverse problem in acoustics.

The eFacsimile project. 
This is a project that we do together

with Google to try to improve how artwork 
is represented on the internet.

it's lead by [INAUDIBLE] researcher Loic 
Baboulaz and several PhD students are

involved in this. 
The questions are how to capture,

represent, and render artwork as well as 
possible.

And to do this, we need some advanced 
techniques on relighting, manipulation

of, the so called, light fields that is 
acquired, and potentially high resolution

solutions for mobile devices. 
There are some demos online that I

encourage you to actually watch. 
because this really doesn't show the

idea. 
This is of course a static version.

But for example one of the demos is you 
take a, an oil painting here.

And you acquire it in such a way that if 
you show it on a tablet and you move it,

it will actually exactly look like the 
original oil painting.

So you get the illusion that you have 
actually the oil painting in the hand.

So if the light changes, the vis, 
visualization will change.

If you turn the tablet, the visualization 
will change.

so then, what is quite stunning and I 
suggest you actually watch it, similarly,

there is another demo which deals with 
stain glasses.

So stain glasses are very interesting art 
objects, but very difficult to render on

the internet. 
And so here we will stimulate the stained

glass, so if you have a tablet in your 
hand and you move it, it looks like if

you had, stained glass in your hand. 
So the tools we use is, we use

traditional cameras, but we also use 
so-called light field cameras.

You might have heard of the light 
[INAUDIBLE] for example.

That's a new generation of camera. 
It's extremely interesting.

And so we need to fully understand light 
transport theory and [INAUDIBLE].

Which uses sparse recovery methods or 
compress sensing.

The website of the project is given here. 
And as I indicated there is YouTube demo

that shows quite realistically the demos 
that we're discussed just in a minute

ago. 
So, next project is about wireless sensor

networks, in particular about monitoring 
visually in a wireless sensor network.

So, sensor networks have deployed for 
many years.

We have large projects here, in the lab, 
on environmental monitoring.

And the current generation is actually 
equipped with camera, and then you have a

problem of compression, of 
representation.

So even though the trend is towards 
smaller and smaller devices, they are

still power hungry and in particular if 
you have a sophisticated camera, the

number of images or the number of pixels 
that is generated might actually overwelm

the power budget of the system. 
And so Dr.

Zichong Chen, who finished his PhD here, 
and did his post doc, together with

Guillermo Barrenetxea, are looking at 
creating large scale sensor networks

equipped with cameras that are energy 
efficient.

So why do we want images? 
Well, here are a few examples.

This is from I think a Berkeley project 
about, monitoring birds' nest.

Unfortunately a snake is showing up an 
he's actually eating all the eggs in the

nest. 
So that's, monitoring for why life

protection. 
Here is an example from the Swiss Alps

monitoring for for avalanche detection. 
Here is also an example from the Swiss

Alps, its monitoring to see weather 
conditions.

And, finally, here is monitoring networks 
that is installed on the PFL campus.

In all these cases you have many cameras 
using small communication devices.

And so compression and representation is 
extremely critical.

So there are a number of results that you 
can find in the thesis of Dr.

Chi Chong Chang, given here in this, 
website, and essentially the idea is

that, cameras can help each other to 
reduce the amount of information that

actually has to be sent to the base 
station or into the cloud, for doing

efficient monitoring. 
So a long with signal processing today,

actually it's moving to single processing 
on graphs.

I don't have to explain to you the 
importance, for example, of social

networks. 
And so Pedro Pinto who was involved here

in the class and is a post doc in the 
lab, together with Patrick Thiran, has

worked on the problem of source 
localization.

So you have some graph here, let's say 
social network and somebody launches a

rumor. 
Here is the source and the rumor gets

forwarded along the edges of the graph at 
different times and you have some

observers, say green nodes here, that 
receives a rumor at some instant of time.

They know where the rumor comes from, you 
know who told you the gossip, and you

know when you got the gossip information. 
So, the question is, you know the

structure of the graph, or you have an 
approximation of the structure of the

graphs. 
You have these observations.

Can you figure out who actually, spreads 
the rumor first.

It turns out this has, an interesting 
solution.

And using only few observers, about 20%, 
you can achieve a very high accuracy in

finding the source of a rumor on a large 
scale network.

And there are many interesting questions 
here, to pursue in this source

localization in social networks. 
And there was a paper that came out last

year, Locating the Source of Diffusion in 
Large-Scale Networks that had quite a bit

of impact. 
The project is actually funded by the

Bill and Melinda Gates Foundation. 
The reason is that one of the

applications is to monitor health 
problems.

For example, here is a map of Cholera 
outbreak in Africa, and the map shows the

river network. 
Cholera is a water born disease, and so,

typically Cholera will actually diffuse 
along waterways.

But you know when people fell sick at 
certain locations and then you can infer

the source of the actual Cholera 
outbreak.

There is another example here, which is a 
simulation of, if you had to figure out

if there was some pollution or attack on 
the New York subway, and if you could

figure out knowing the network of the New 
York subway and when you start detecting

the problems where the source of the 
problem actually was.

The next project is on sampling, so we 
have worked on a new theory of sampling

here called Finite Rate of Innovation 
Sampling, and it is used in

communications problems, and in 
monitoring problems to reduce the number

of samples being transmitted or acquired. 
Dr.

Freris, who is a senior scientist with 
doctoral students and MS assistants, are

actually working on doing ECG monitoring 
at very low power for wireless health

monitoring. 
So here is a block diagram, it's

relatively complicated so let me not get 
into this, but it uses some fairly

sophisticated techniques to reduce the 
sampling rate so as to reduce the energy

consumption on these wireless devices. 
So, this project is actually sponsored by

somebody well known, Qualcomm, interested 
in the theory of sampling.

And the extension here for this 
particular project has been

generalization of the initial finite rate 
of innovation sampling methodology.

To get better compression, and better 
modelization of the signals.

So here we have the ECG signal, and then 
there is sophisticated models that

allows, to take very few parameters, to 
model the ECG signal.

There are a number of papers here, the 
initial paper on finite rate of

innovation sampling is this 2002 paper, 
and the number of recent papers have done

extension to this theory. 
So if you like sampling I welcome you to

actually read up on this stuff, it's one 
of my favorite research topics.

When we talk about sampling already in 
sensor networks we have mentioned that

placing a sensor is like taking a sample. 
And so that spatial sampling, now if you

do spatial sampling, you can also use 
mobile sensors and Dr.

Unnikrishnan here, a post doc in the lab, 
has worked on this or generalization of

the theory of sampling when you have 
mobile sensors that can actually go over

a field in an arbitrary fashion. 
Then you maybe show this in an example.

It's again a temperature monitoring 
example here on the EPFL campus, or you

have buildings. 
You have that open space between

buildings. 
Those buildings are, of course, hot.

The open space are cool. 
And you would like to have monitoring of

this temperature field not with static 
spatial sensors, but with people running

around, having a thermal meter let's say 
on their mobile phone.

And the question is, how accurate can you 
actually measure temperature using a

device like this? 
And so, this is being done actually for

pollution monitoring in the city of 
Lausanne so there's some equipment put on

buses to measure pollution parameters. 
And what we do here is we try to develop

a theory of how good you can sample when 
you have these mobile sensors going over

a surface and measuring a field. 
The results are very mathematical but are

interesting because our non-trivial 
extension of sampling theory through

multiple dimensions. 
And a few papers are mentioned here if

you are interested in more detail. 
The next project is about a new way of

doing image acquisition. 
So in this class, we have seen sampling

and we have seen quantization. 
And when we do quantization typically we

say, let's take [UNKNOWN] samples and 
then take as many bits as possible.

Let's say eight bits for speech, 12 bits 
for images 24 bits maybe for audio,

etcetera. 
Now here we took the extreme other

example we said lets build an image 
sensor that has many, many, many pixels

but the pixels only detect either a 
enough light or not.

So the pixels are actually binary 
detectors.

And so you have a light intensity here. 
Which changes over space.

You have a lens that smooths the light 
intensity.

So what reaches the camera is this smooth 
curve here.

And this smooth curve you sample very, 
very, very finely.

But you only decide if it's above or 
below a certain threshold.

So the sensor only generates a sequence 
of binary digits.

So that's the imaging model. 
And this has been studied by Dr.

Feng Yang, did his PhD thesis on this, is 
now a post-doc working on this project,

and a whole slew of other people. 
This was a very extensive project.

And what is interesting is that this new 
way of acquiring images, for example,

allows to do high dynamic range imaging. 
Here is a simulation of a high dynamic

range image in a much easier way than 
with conventional cameras.

That's one advantage. 
Another one is that you can have very,

very cheap sensors. 
So here's an example of one that was

built in the lab. 
And then, you take many, many frames.

They are extremely noisy. 
If they look noisy, they are simply

binary, so you only have zeroes and ones, 
but you have enough of these, and you do

an optimal reconstruction method. 
You actually can recognize here, the logo

of EPFL. 
There are publications here that you are

welcome to look up. 
And the thesis is online.

Last but not least Rambus silicon valley 
company, actually works with us on this

and has acquired some of the technologies 
that was developed in this project.

And old classic is trying to predict the 
stock market.

So, we gave it another shot. 
so Lionel Coulot did his PhD thesis, was

co-advised with Peter Bossaerts who is at 
Caltech.

And we were trying to understand if 
methods from information theory would

allow to predict models for the stock 
market, and that requires statistical

models for what the stock market might 
be.

And what is interesting is that you have 
to decide between very sophisticated

models that might be overkill and are 
hard to estimate, and very simple models

which might be too simplistic, but which 
might be very robust to things that

happen in the stock market. 
And, in the end we used coding theory and

classic algorithmic methods like dynamic 
programming to come up with a method that

decides what is the correct model at 
every time of, the observation of the

stock market. 
So I'm just going to show a picture.

And the picture is, is a value on the 
stock market.

And the question is, can you detect if 
the stock market is in a bear market or a

bull market? 
So when the stock market goes up it's,

called bull market. 
If it goes down, it's a bear market.

What is very hard is to decide by 
watching every day what's happening.

If currently the trend is going up or the 
trend is going down and you need to do

this with an online algorithm. 
Okay, you cannot look into the future and

this method developed by Lionel allows to 
do a model fitting and to very quickly

detect when the stock market changes from 
a bull market to a bar, bear market.

The thesis online and this was sponsored 
by, as you may guess, by a bank.

And the results are interesting, but we 
are still having a regular day job so you

can guess that the method is not 
completely fool proof to predict the

stock market. 
But the methods, the algorithms, and the

theory behind it is quite cool. 
The next few projects are so called

inverse problems. 
So inverse problems are problems where

you have some measurements but the 
measurements do not describe the signal

you're interested in. 
But some indirect measurement of the

signal, so you try to invert the system 
to go back to the original signal.

You all know about computerized 
tomography, a medical image method, where

you can see inside the body without 
really going there.

And that's a typical inverse problem. 
Here we are interested in inverse

problems in environmental monitoring. 
So, the first example is diffusion

equation. 
And we have a physical phenomena, for

example temperature has been discussed, 
or atmospheric dispersal of pollution.

We want to measure the field at locations 
where we can put sensors, and the goal is

to find where are the sources, for 
example, of pollution.

Now this is a hard problem because, you 
have to model how, for example, pollution

is being diffused. 
That depends on weather patterns and so

on. 
But the tools we are using are typical

signal processing tools, for analysis. 
Sampling theory for exemplifying finite

rate of innovation sampling or 
compressive sensing, that has also been

mentioned earlier. 
Let's look at the picture.

That's a very simple example of this. 
Assume you have two smokestacks and

inside a factory compound, and the 
smokestacks produce pollution which

changes every day. 
You don't know how much pollution is

being released, and you're working for an 
environmental monitoring agency.

You put sensors outside of the compound 
and you measure what arrives, in terms of

pollution, at these sensors. 
And the goal is to figure out if what

came out of smoke stack was within the 
bounds allowed, lets say by z,

Environmental Protection Agency. 
So this is a interesting and non-trivial

problem but there are some interesting 
results that were produced by Yuri

Ranieri, whom you all know because he was 
the famous Master Chief assistant for the

BSB class. 
So we are able to recover sparse sources

using this inversion method. 
we use this finite rate of innovation

sampling techniques to actually do it. 
And here we is a list of publications

that came out of this research. 
This problem is also an inverse problem.

It's a Fukushima inverse problem. 
It is a PhD project of Marta

Martinez-Camara, and a few other of us 
are involved in this, and we collaborate

with a specialist Andreas Stohl. 
Who is a specialist of monitoring of

radioactive diffusion. 
So what we like to do is figure out how

much radionuclides were actually released 
in Fukushima at the time of the of the

nuclear accident at Fukushima. 
We have only very few sensors, they are

located around the world very far away 
from Fukushima.

And the question is, is it possible from 
these few measurements around the world

taken later, to invert the entire process 
as I diffuse the initial release of

radioactive material into the atmosphere. 
What tools are we using?

Sparse regularizations, so that's 
compressed sensing.

And we need to using atmospheric 
dispersion model to understand how

radioactive material from from Fukushima 
was actually transported across the

world. 
So one result that we have and which is

very interesting is we were able to 
estimate the emission of Xenons, that's

radioactive gas that was released at the 
time of explosions at Fukushima, went up

into the atmosphere, was transported by 
weather patterns all over the world.

And from the measurements all over the 
world, we were able to pinpoint exactly

when the Xenon was released, and how much 
Xenon was released into the atmosphere.

And it turns out we actually know the 
total amount of Xenon that was released,

because after the accident no Xenon was 
actually left in the nuclear power plant.

Currently we're trying to go beyond this 
and estimate the Cesium release, but that

turns out to be a harder problem. 
The paper that describes this will be

published ICASSP this year and is 
available online here in infoscience.

Last but not least is a project we call, 
"Can One Hear the Shape of a Room?" It's

a PhD project of Ivan Dokmanic and 
several other people in the lab, in

particular, Reza Parhizkar, Andreaz 
Walther, have worked on this.

And also we have a collaboration with Yue 
Lu.

He's now with Harvard. 
Now you know about this problem because,

Ivan gave module 512 about gear 
dereverberation, echo cancellation.

And, uh,the next step is to say, if I 
listen to echoes, can I actually

understand what is a room shape? 
So if I know the room shape, then I know

how to generate the echoes. 
But if you give me the echoes, can I know

the room shape? 
It's a classic inverse problem, very cute

one. 
And we usually explain it by saying,

let's say you enter a room, you're 
blindfolded.

And so you don't see the room at all. 
You snap your finger.

You therefore elicit echoes, you listen 
very carefully to the echoes.

Can you exactly see or hear the shape of 
the room?

Now this has a beautiful theory, which we 
won't have time to really explain, but

that you can read up about because it's 
published material.

But if you have a source or receiver you 
have a direct pass between the source and

the receiver, and you have echoes given 
by the walls.

The echoes given by the walls correspond 
to so called mirror or image sources, so

this is the same as if you had a source 
here and the sound would have gone

straight here. 
So if you can locate all these image

sources, then you can actually locate the 
room.

The walls, therefore the room. 
And this is, you know, in principal

do-able the question was is it always 
true that this can be done?

And is it also realistic to do it in 
practice?

So, here are examples of a system with 
five microforms, you have one source five

microforms. 
You have somebody snap his finger and you

have the echos related to the walls and 
you see there is a complexity which is,

these echos come in random orders because 
different walls are at different

distances of the microphone. 
And the question is, can we find out the

shape from a set of measurements as we 
see here?

How many measurements do we need? 
Can we have a robust algorithm?

So the answer is summarized in, yes we 
can.

And there are some experiments we did, 
both at the labs.

So this is one of our seminar rooms we 
created a, an artificial wall here to

have different shapes of rooms. 
So this is a typical shape of room.

Then in this case, with five microphone 
and one source, we were able to estimate

the size, shape of the room very 
accurately to more, better than 1%.

And once we had this, we said, well, 
let's see how robust this is.

We went to Lausanne Cathedral and that's 
actually a foyer of the Lausanne

Cathedral, which is not at all needing 
the assumptions of the algorithms that

I've described very briefly here. 
And it was still possible to see the

major refractors, meaning the major walls 
here in the Lausanne Cathedral.

And so the answer is yes, one can hear 
the shape of a room.

And you can visit Ivan's web page to see 
more details.

Now these were just a selection of 
projects, of works that is being done by

PhDs and post-docs and senior researchers 
in the lab.

Please go to the website, as that gives 
the entire portfolio of research here of

what the lab is currently doing.