36C3 - phyphox: Using smartphone sensors for physics experiments

36C3 preroll music

Herald-Angel: Good. Ladies and gentlemen,
we have here a talk by Sebastian Staacks.

Do I pronounce this well?
Sebastian Staacks: Yes.

Herald: Yes. Staacks. Staacks. [In German]
Ich musste das mal in Deutsch sagen. And

he's related to the University of Aachen.
He did a PhD physics. And he was in a team

that developed a fantastic application, as
I mentioned earlier on. He developed the

app phyphox. Do I pronounce this well?
Staacks: I would say phi-phox, physical

phone experiments.
Herald: Okay. Yep. Of course. I'm sorry.

I'm not in that kind of department. But
this application actually gives you all

the possibilities off your the usage, off
your smart smartphone. Really? Really

extending certain borders, to my opinion.
So please give a warm, warm welcome here

to Stefan.

Applause

Stefan: Thank you. Thank you for the
introduction and welcome everybody to my

talk. Yeah. As you've just heard, I'm a
physicist from the RWTH Aachen university

where I developed the app phyphox. Phyphox
is an app for those of you who do not know

it already. That uses the sensors in the
smartphone for physics teaching. So the

idea is that students can use their own
phones to do experimentation in class, in

the lecture hall. So for schools and
universities. I should explain. That in

contrast to some other talks by me. This
one will not be that much about education

because it is the chaos communication
Congress and this is the hardware track

here. So I tried to tell you a little bit
about the app, a little bit about the

sensors that we have on our phones and.
Yeah. Would we love to get in touch with

some, especially people from maker
community and from open source communities

to find some connections, how he can get
many open source projects together?

Because I've got so much feedback from
teachers and I think I could also use some

feedback from other developers as well. So
I would like to start with a short

explanation of what we actually do. So
yes, I said I come from a university and

there we have this introductionary lecture
for physics students, which is called

experimental physics one. And it's typical
lecture. Looks like this. We have a fancy

new lecture hall by now, but the situation
is the same. We've got 300 I think 370

students this year sitting in a lecture
hall and doing no experimentation at all.

There's only one guy experimenting and
that's the professor. And the students are

sitting there and enjoying the whole show
like they would enjoy a YouTube video and

maybe they are mildly amused if something
goes wrong. OK. And we thought we could

change this by using the sensors in the
smartphones. We're not the first ones with

the idea to use the sensors there, but for
some reason we decided to write our own

app, which turned out to be quite
successful then. So in contrast to the old

version where students just had to look at
and I'll get the assignments where they

can do their own experiments with their
own measurement devices. And to give you

an idea of what this looks like. I would
like to start with the first experiment.

Which is about centrifugal acceleration or
centripetal acceleration depending on your

preferred frame of reference. So the idea
is from a rotation movement, we want to

measure the radial acceleration as a
function of the angular velocity. So the

rotation rate. To do this we take a
regular smartphone, this is an iPhone 8 in

this case and we put it into a salad
spinner. Okay. We get some rotation in

there and whoops let me just place it in
there. Sound is not important, but it

sounds nice. I have been told. So here we
get the live data from the phone already.

Acceleration on the y axis and angular
velocity on the x axis. If the salad

spinner is actually moving. And what you
see is the faster I rotate the spinner,

the farther on the right you get your data
because that's angular velocity and also

the radial acceleration increases. If I'm
not going too fast because then I do not

get any data at all anymore. Let's slow
down again and we can fill up the gaps

there by going really slow and filling up
this path. And in the end, if so, who here

has a physics background some more than
expected. Great. Because those of you who

just raised their hands would not be
surprised that we expect a square

relationship between the radial
accaleration and angular velocity. Those

of you who do not know will believe me
from this plot where on the x axis we've

got the angle of velocity squared and on
the y axis the radial acceleration we get

a straight line and that's what you would
expect. So besides the physics, because

this is not that much about the physics.
This is a simple experiment all our

students could do and actually they ge, we
gave them this assignment. We gave them

also a bonus point if they created a
video. Don't worry. Their consent to that

we use the video was not related to the
point, they first got the point and then

we asked for their consent to use the
video. And we learned two things from

these videos. A Our students do not really
have salad spinners. they've got bicycles

and office chairs, but b and that was the
most important thing. It looks like I

mean, these are from this year where we
got almost 100 videos they we actually

could trigger them to go out, search for
something where they've got the

rotationary movement and they could repeat
this experiment. Ok. Another example which

actually changed just the course of the
lecture a little bit is a situation where

we first give the assignment before we
actually let them, before we actually

discuss the theory behind this, which
means in this example, this is a little bit

older because we did not get there yet this
year, we assigned our students to build

string pendulums. They look very similar
because we were very precise about how

they should build them. And then we had an
online form where they could submit the

length of their pendulum and the frequency
they received from it that they measured

with the pendulum. They should do this for
three different cases. And the idea was

that we did this assignment long before we
discussed the pendulum in the lecture so

that they have got a little bit of
research experience. And after we

collected all the data from them, then the
lecture would discuss the pendulum. So the

physicists were there now. We do a small
angle, approximation solving differential

equation. All this theory stuff. And in
the end we were done, we could tell our

students, well, we do not have to do this
experiment on stage. Now, because all of

you did this experiment and we simply can
compare the theory that we just arrived

with your data. And it worked out quite
well. So you see most of the white points,

which is the data from the students
matches the theory, which is the orange

line, except maybe for those three who
should proceed on a career of theoretical

physics. But yeah, so this is all
something got nice feedback from and this

is in principle how we use the app and
what it's designed for. There are also of

course many applications in school by now.
More teachers use this in school than we

use it at the university. So we take this
into consideration as well. But that's the

reason that I am standing here talking
about the sensors in the smartphone.

That's the reason that I am trying to
access them. so let's have a look at what

sensors we actually have in our phone. I
think the first one that most of you would

think often talking about sensors besides
obvious stuff like the microphone would be

the accelerometer. So I think yeah, I
think I first explain how the

accelerometer works. OK, so the
accelerometer in your phone is actually a

so-called MEMS device. MEMS is M E M S
stands for Micro Electrical Mechanical

System and it looks roughly like this.
It's a simplification. If you search for

actual MEMS devices, simply search for M E
M S and accelerometer and you find some

pictures. They usually are a little bit
more complicated, although the

accelerometer is not that much more
complicated. It consists of an orange

case. Yeah, well so far so obvious, but
also two contacts. The blue and the red

one and important part is this silvery
structure here or the metallic structure

which is under etched its bit hard to see
on this picture, but it's actually

floating. It's only attached to the sides,
you see light in between here. So if you

move around, the accelerometer the inner
path, can actually move. So let's do this.

So at each point where the device is extra
riding in one direction or the other

direction, due to inertia the that the
metallic part in here is distorted, moved

into one direction and we can measure the
amount by which it is deflected by this

movement with the two contacts by
measuring the capacity between these

structures. So that's the principle of the
accelerometer. One thing to mention at

this point is that it's in the sense of
physics. It does not really only measure

acceleration. It measures acceleration you
see in this image of the device

accelerating. We get some data, but if you
imagine we take this device and rotate it

like this, then of course you also get a
deflection of the of the metallic part by

gravity. So gravity is pulling it down as
well. And that's the main reason the

accelerometer is in there because the
developers and manufacturers of the phones

are not really interested in measuring
acceleration, at least there aren't that

many use cases for it. But instead, what
they want to have is an indication on

which direction is down or which direction
is up. So when you rotate the screen of

your phone, actually they can rotate the
content of the phone as well or with this

you can also then control video games by
tilting your phone and stuff like this.

Because gravity also deflects the
accelerometers. Earth's acceleration,

which you try to avoid because from
didactic point of view, this is a

nightmare to distinguish these both. But
the point is that we can detect rotations

like this and this is pretty much in every
phone. I mean, this is not really a

statistic. This is just the first pie
chart we have about availability. I have

never encountered a single phone or tablet
that does not have an accelerometer. So if

anyone ever encountered some special
device, some very unique device that

doesn't have one. Let me know because I
would be interested in this at least. I do

not know of any device on which phyphox
actually runs, which doesn't have an

accelerometer. A bit more interesting is
which data rate we can achieve. So most

accelerometers have several hundred
samples per second. Actually the fastest

ones go up to 500 hertz and but there are
also many devices that only do one hundred

hertz That's 100 values per second. These
are mostly the cheaper Android devices and

all the iPhones. So I think the internal
accelerometer will do more on an iPhone.

But I have to admit, at some point I can
understand why they might limit this. But

on an iPhone, you get 100 hertz. That's
the limit. From the API, what you can get

there. But this is actually quite a lot. I
will later see what we can do with this.

And one other point about calibration of
this thing. Actually for all the sensors

to get reasonable units from the system so
the acceleration is given in meter per

square second. I just realized that if I
get the units, that's something I would

really tell my students. But yeah. So on
the x axis, it's a meter per square second

and you see that as a wide range of values
that you get there. So this data is from

our sensor database. I would mention it
later as well. This is contribution from

our users what data there this. This only
absolute value that we get from resting

phones and we would expect nine point
eight one meter per square second for

earth acceleration. There are some local
variation, but not on that scale. So do

not expect your sensors to be well
calibrated. Also, if you've got any app

that tells you you can push a single
button and then calibrates your sensor,

don't trust it. It's not that simple.
These sensors may have different errors on

each axis. They're all 3D sensors we've
got an X, Y and z axis. These errors can

be linear errors so you have to multiply a
correction. It could be an offset. So it

would have to add an correction. And on
top of this, the entire device could be

tilted within your phone. OK. So actually,
if you look into the data sheets of the

accelerometer, they have some tolerance on
how much they might already be shifted or

rotated within the package. And when
soldering it into the phone, I would

assume there will be an additional error.
I've seen so many different errors on

different phones. It's not that easy to
simply calibrate that. But let me give you

an example of what you can do with it. Or
just a quick look first. So we see in our

app. Yeah. So this is phyphox. OK. Thank
you. Got this. You have an entry

acceleration with G. That's the extra raw
data from the sensor or as raw as we get

it. If I started you see if I shake it,
you get some readings there. It's fast.

It's already great. You can apply to
pendulum and measure the acceleration of

the pendulum like this. But something I
want to demonstrate is that we can also

get the frequencies from this data by
doing a fourier transform and calculating

the frequency spectrum of this exploration
data and to demonstrate this I brought a

little device a old hard disk drive. It says
it's broken, but it's still rotating and

that's important part for us. So if I
place my phone on top of it, start the

measurement. Turn on the hard disk drive.
And then you see a peak showing up in the

spectrum and it settles at 120 hertz. If
you don't believe me. Unfortunately, we

don't have a camera here right now. You
can later have a look. It's supposed to

run at seven thousand two hundred RPM,
which is 120 hertz. We can even get a time

resolution of this. So if I turn it off
again, you see how the frequency drops

down. And if I turn it on again. There it
comes up again. OK. So this an example of

what you can do. It's great for students
that can check if the washing machine at

home is working properly or they can
check other things. But usually I do not

like to bring washing machines to talks.
So I used the hard disk drive here. One

other thing you might have noticed before
is that we've actually got acceleration

with G and acceleration without G. The
second one is actually a sensor that

removes Earth's gravity. So if I start the
one with G, you will notice that down here

on the Z, the axis you still have the 9.81
meter per square second, which is great

because if i rotate the phone. This
contribution goes to other axis and we can

determine the orientation of the phone.
But this is bad actually for dedactics

because actually the phone is resting.
It's not moving at all. There's no

velocity involved. There's no
acceleration. So luckily, there's also an

acceleration without G, which gives us
roughly 0 an all axis unless I actually

accelerate this thing. Problem with this
is this is only a virtual sensor. This is

a sensor that's fusing the data from the
accelerometer with an additional sensor

like the gyroscope. So we can actually
distinguish between rotating the phone or

accelerating it in one or the other
direction. Usually you only get

acceleration without G. If you also have a
gyroscope in your phone, I've seen two or

three devices that offer you acceleration
without G, even though they don't have a

gyroscope. This case, don't trust them.
This is merely guessing. OK. So it's. They

probably have only low frequency filter
on top of this, or they're averaging out

your movement and this doesn't really work
for anything. Yeah so that's the

accelerometer or one other thing I want to
mention is if you look into the API to

access the sensors yourself for some
reason you will notice acceleration

without G is usually called linear
acceleration in our app since it's made

for teaching. We decided to call it with
and without G. So if you find

accelerometer, that's the one with G and
linear acceleration is the one without G.

If you look at other apps or the API.
Okay. Next up, I already mentioned this

one is a gyroscope. If you have, some
physics background. Then when you think of

a gyroscope, you're thinking of a device
that's spinning fast so it has some angular

momentum and then usually you want it to
be heavy and to have the weight at the

large radius. We've got a strong moment of
inertia so that you get when it's spinning

fast, a strong, angular momentum and due
to the conservation of angular momentum.

These spinning devices can keep an axis
regardless of rotating the frame around

it. That's what I was thinking about, a
gyroscope of what I think is a gyroscope.

When you just give me the term out of
context, of course, a heavy, huge, fast

spinning device is the last thing you want
in your phone. So that's not what's meant

with the gyroscope when people are talking
about gyroscopes in your phone. Instead

there again you have a MEMS device. So
again, micro electromechanical system. You

notice this looks almost exactly like
accelerometer. If you look for real

devices, those are actually much more
complicated because they need some

specific geometry to make sure that they
do not act like an accelerometer. But the

principle is easy to explain with the same
geometry. So we again have this floating

metallic part and we've got 2 contacts. So
again, we've got a part that can wobble in

this direction here. But on top of this,
we've got the motion that's perpendicular

to this. So this is now not depicting the
motion of your phone, but this is

depicting a vibration that the gyroscope
does by itself all the time. So there are

different ways to build them. Some have a
rotary motion, some have this linear

motion. Also, the way to create this
motion makes this device so much more

complicated. But in principle, it's a
similar structure which is vibrating forth

and back and now if you add rotation to
it. It's a little bit hard to see it as

it's rotating the inner part now suddenly
gets deflected. That's changed, right?

Frame of reference. So let's get the
camera in sync with this device. What you

now see is that the inner part is moving
left and right, although the device itself

is only moving up and down. And the reason
is I don't want to deduce it entirely

here, but most of you probably have heard
of it. This is the Coriolis effect. So,

yes, in fact, your phone is determining
the rotation rate of your phone, not the

actual angle, but the rotation rate or
angular velocity due to the coriolis

effect, which is just mind blowing if you
do some of the calculations. There are

some manufacturers on the Internet which
claim that they can detect a movement of

the order of magnitude of a single atom.
And I believe them because we use similar

structures in solid state physics. So
that's possible. If you want to try it,

just turn on the gyroscope on your phone.
And do slight rotation like this, which is

about the Z axis, one perpendicular to the
display, you can detect really slow

rotations with this. And think about the
fact that this is done using the coriolis

effect and it's just mind blowing I think.
So this sensor is a bit more available.

Actually, almost 80 percent of the phones
have them. This has become significantly

more since Pokémon GO. The reason is
when this game came up, suddenly people

noticed that there's a device called the
gyroscope. And if it's not present, they

did not have this AR mode where you can
actually take pictures of the nice cute

Pokémon and so on. So this is when the
many people noticed it and the

manufacturers decided, OK, let's just
throw in the gyroscope as well, because

it's not that expensive, in fact, usually
it's on the same chip as the

accelerometer. Then they're sold as one
thing it's an IMU - Inertia Measurement

Unit not important at home, but so it's
quite a common thing. And the sensor rates

look pretty much the same. You mostly
notice the dip in the 100 hertz regime

because those are the real cheap phones,
which then also don't have a gyroscope.

But most of the phones achieve higher
rates. Again, since we were laughing

before the iPhones also are here again at
the 100 hertz. Wouldn't make sense to have

the gyroscope faster at this point. Yeah,
but that's it about the gyroscope you've

seen it in action in the salad spinner.
And that's one of the sensors you do not

really see that often directly, but were
just mostly there to assist other things

that you do where you need to get smooth
motion like controlling games, AR . And

actually removing the Earth's acceleration
from the accelerometer. Next up is a

magnetometer, which I think is a more
obvious sensor because that's your compass

in your device. So when you're doing
navigation with a GPS in your car, it's a

simple thing. GPS gets a position, you get
a sequence of position as you going and

from the sequence of the positions you
get, the direction you're moving in your

car and your phone is attached to the
dashboard at least i hope so. So it's

pointing in the same direction you're
moving, everything's obvious. But if

you're standing on an open space looking
for not sure a train station or anything

and you wondering which direction you want
to go from point of view of GPS, it's

always the same position it doesn't get an
orientation. You need a compass, which is

the magnetometer. How do we get a compass
on your phone? This is usually a hall

sensor. A hall sensor is in principle just
a conductor with charge carriers so these

are the nice shiny white balls here
drifting from one side to the other so

it's just an electric current. And if you
apply a magnetic field to an electric

current or to any electric charge, then
there is an effect. You might know from

school, which is called the lorentz
effect. So there is a charge going one

direction, you get the magnetic field
perpendicular to this and then the charge

is deflected into a direction
perpendicular to the flying direction. And

yeah, that's lorentz effect the older
guys, of you would know it from CRTs. If

you bring a magnet close to a CRT, the
entire image is messed up due to this

effect. And that's what we're using in
hall effect sensor or hall sensor you've

got this electric current and if you bring
a magnetic field close to it, the charge

carriers are deflected to one side or the
other. And therefore, if you're measuring

the voltage perpendicular to the flow of
the count, you get. Yeah. You get an extra

voltage that's proportional to the
magnetic field. That's the hall effect.

That's how your phone is able to determine
the magnetic field. This one is even more

common than the gyroscope simply because
it's used for navigation and people start

to notice if it's not. If it's not present
and they do not get an orientation in the

navigation software. But the actual rate
of the sensors is much slower than for the

accelerometer. Most of them are running at
100 Hertz. It will be important in two

more slides. Besides that, there's not
that much strange about the availability

of this, but it's extremely sensitive
because it's supposed to measure Earth's

magnetic field. Earth's magnetic field has
the strength around 50 micro Tesla. This

is not much actually if ever carried the
magnetic magnet with you. Did you fear of

some force from the Earth's magnetic
field? Of course, it didn't need to build

some compass where the needle is floating
on something like this to actually get a

rotation. It's a very weak field and
that's good news and bad news as well,

because on one hand, it's very sensitive.
downside is it's very sensitive. Which

means it saturates very early. If you want
to measure the magnetic field of an actual

magnet. Don't even try it will saturate
right away. You do not get anything to

demonstrate how how sensitive this
actually is. I've brought a flashlight, so

a very simple one. And I switch to a modus
where we've got an s.o.s signal. That's

coming up, a point in this direction and I
place it next to the magnetometer in my

phone. And yeah, you see right away so
much of his seeing the lights are pointing

in this direction. You see the s.o.s
signal popping up in the magnetic field

reading simply because of the current
going through the LED. So that's what we

call an Oersted-field. This is just the
typical magnetic field you get from any

current flowing. So I stop it. We got a nice
SOS signal over there. Three short, three

long and three short signals. And it's just
coming from this simple flashlight. And

this is also a good indicator on how
sensitive this thing is. I mean, if you

place your phone in a case with a magnetic
some magnetic closing mechanism, compass

wouldn't work anymore. If you're not
careful when paying your clothes and you

place your phone on the big magnet that
removes the theft protection from the

clothes, something in your phone would get
magnetized and would certainly be stronger

than Earth's magnetic field. For the rest
of the day, your compass would be pointing

in the wrong direction. Okay. Luckily,
usually the phones are able to notice this

and they recalibrate the phone to simply
subtract any constant fears. That again is

bad. If you want to do absolute
measurements because you have not much

control over the recalibration mechanism,
you can access the raw data value. So if

you folks there's a checkmark where you
can disable the calibration, but then you

have to do everything by hand. You will
certainly have some background that's

annoying. And one other thing, you should
also take care and notice where your

actual magnetometer is because in most
phones it's on top left corner, top right

corner, top center. And this Pixel 3 is a
very strange one. It has it on the right

hand side, but it's never dead center. I
think because of all the currents in the

phone, I mean, you're charging your
battery with three amps. How much you

charge them now? This would yield a
stronger field than a flashlight and you

would see it in the magnetometer again.
Now for what you can do with this. So as

little homework for all of you who came by
train yesterday, when I came here on the

ICE, I turned on the magnetic spectrum,
the same thing as the acceleration

spectrum you just seen. And when you're
doing it on train, you would see a peak at

16.6 hertz. It might depend on your actual
seat. You might move it around a little

bit. But so far I usually always saw this
peak. This is the electrification

frequency of the German railway. So you
can simply check if it's working properly.

You should see 16.7 hertz. Okay. One other
thing that some of you might get in your

head right now, that you could do this
with simple electrical outlets. There you

would get a problem with the rate. So
that's what I mentioned, that the rate of

the sensor is quite important. I also got
something via Twitter yesterday. Just as a

response to the other one, I thought,
well, I was looking for an example like

this for this talk talk, so I just put it
in. This is a measurement of an American

power outlet which is run at 60 hertz. But
this guy is seeing 40 hertz and he was

wondering about this. That's what's called
aliasing. So the alias effect, sort of you

might notice this from computer games.
They usually use it in slightly different

context. The idea is if you're measuring a
frequency that's higher than half of the

data acquisition rate of your sensor. So
this one is runningat 100 hertz like most

of the phones do. Then half of this
frequency is what's called the Nyquist-

frequency. And you notice that the
spectrum goes from zero to the Nyquist-

frequency. This is simple math, not simple
math, but its maths. The roots of the

fourier-transformation, you could say so.
And if you try to detect a frequency

that's higher than this, so an American
power outlet with 60 hertz, actually the

higher frequency is showing up as on the
other side of this upper limit at 40

hertz, even if you go to a higher
frequency, it would shift down further and

further until reaching zero and then it
would shift up again. So if you're

interested in this. Check out some
articles about aliasing. If you're not

that interested in this. Just keep in
mind, if you're measuring frequencies that

are higher than half your data acquisition
rate, you will not see the correct

frequency. OK. Then one of my favorite
sensors, the pressure sensor for this one

I need. Again, the phone. That's not on a
wire. Let me before before I show

anything. Let me demonstrate what it can
do, because that's something I find quite

surprising. Let's turn on the measurement.
By the way, those who are wondering how

this works. There's a function in phyphox,
we call it remote access. It's basically a

web server running in the app which
provides the data so we can simply access

the data on the phone to demonstrate or to
control the measurement. And now here we

see the pressure sensor. Right now, just
mostly noise or what I do now is I hold it

up. And if we wait a few seconds, you
would see that the pressure's actually

dropping. It has dropped far enough. Then
I place it on the ground and the pressure

is rising again. So actually, your phone,
if it has a pressure sensor, has a

pressure sensor that's sensitive enough.
So we turn it off to measure a change of

pressure of a distance like this. OK. And
that's again, when I first tried this, I

repeated this test several times before,
believed it was just not by accident. And

how do they do this? You have got another
device that actually has a cavity. So

below the bluish gray part, there's a
cavity in there which is covered by a

silicon membrane, which is the bluish
part. And if you change the pressure this

simply moves it like you would expect from
a membrane just in small. And to detect

this movement, here is some material on
top of this which changes its resistance.

Or resistivity depending on the strain
created by morphing, dismembering. And

unfortunately, this sensor is not that
much available. So about a third of the

devices that we know of have the sensor.
Of course, there's some bias in there from

the users that submit data to us. This
means that, yes, these are usually the

more expensive devices. So my rule of
thumb is if it's an iPhone, they usually

have the pressure sensor except for the
iPhone SE or some older models. If it's an

Android, if you payed half as much as you
paid for an iPhone, then you have a good

chance that you have to pressure sensor as
well. But OK, that data rates? Yeah.

Varies a lot. So the iPhones, like you
just saw the rate of about 1 Hertz. Most

Android phones are on five, ten or twenty
five hertz. I've never had a device like

this in my hand. It does 100 hertz. I
don't really believe that this makes sense

because I already noticed on my phone that
I think it does 25 hertz. Just handing it

because of the sealed casing introduces
more noise than you can actually use, at

least for these small distances that I use
it for. But you can do other funny things

with this. So this is something I received
by Dianna Cowern. You might know her as a

YouTuber called "The Physics Girl". She
used a pressure measurement on the flight.

It's something you should do anyways,
because that's the way you can figure out

how much air you get to breathe up there.
It's much lower than you might expect.

But she saw something else. So at some
point she saw the drop in the pressure and

increase again. And she asked her
followers, what could this be? And I'm not

asking the audience right now. I just give
you the solution. She wasn't lavatory and

she flushed the toilet. So when water and
air gets sucked out, you can actually

measure this. And then about a month ago,
I found someone else who allowed me to use

his measurement. So this guy, Phillip
Smith, was on an airplane again. But he

did not actually go to the lavatory. He
stayed on his seat and he just checked

when people were flushing the toilet. So
as he sat, there was there were

turbulence. So they couldn't go for a
while. And then there was the rush while

the toilet and he was plotting it. So just
for those of you that came here by plane,

just a hint as a conversation starter next time,
when the guy next to you goes to the toilet and

he comes back, tell him exactly all of the
head to flush the toilet and ask him why.

Okay. And you would enjoy the rest of the
flight. Some other example that we

actually use is measuring the movement of
an elevator. So this is a lift in Aachen.

We have the accelometer which measures the
acceleration of this thing, gets the total

height difference of the elevator from the
again, from the pressure sensor, a

barometer. That's a pressure sensor. And
the velocity of the elevator as well from

the change in height. OK, so next time you
enter an elevator, I want to see you all

to take out your phones and measure the
distance that the elevator is traveling

and the velocity at which it does so. OK.
So these are, in my opinion, most

important sensors, some honorable
mentions. Almost all phones have a light

sensor as well, which controls the display
brightness depending on the ambient light.

Unfortunately, there is no API on IOS
to access this. So if there are apps

that seem to access a sensor like this,
they usually use the camera instead, which

is which also works quite well. But it's
slightly different since the difference

between illuminance and luminance, which I
do not want to go into detail here. And on

most Android phones, they are badly
calibrated or do this so much difference

in the quality of the sensors. We have to
check it on your own phone if it's worth

anything. But it's a bit difficult. This
proximity sensor, which is the one that

turns off the screen when you hold the
phone to your ear when you're

actually doing your call. Sounds
interesting, but unfortunately it only

distinguishes or has I know it
distinguishes between between the near and

far value, which is the difference between
five centimeters. So I do not have that

much use for it. There is the temperature sensor,
maybe if they are officially there, then

they usually come along with the humidity
sensor, but that's the sensors in your

phone. So you should be a little bit
skeptical about this. You're mostly

measuring the heat from your battery or
from your device. They tried to compensate

for this, but that's a difficult thing to
do. So if you actually, one, need a

thermometer, take a thermometer. They're
not that expensive. OK. You might see some

temperature sensors that are not official.
Which phyphhox can pick up. Those are usually

temperature sensors that are part of the
pressure sensor to compensate for

temperature effects. So they're not even
designed to get an outside temperature.

OK. So I wanted to mention this. While the
information about where we got the

information about the sensors from, so in our
App at the very bottom, does this entry

submit to a sensor database which tells
you to leave the phone resting on a table?

It also checks if you're actually doing
this, doesn't let you submit it before it

is happy about the error rate or the
standard deviation of the accelometer. And

if you submit it, we collect the data on
phyphox.org/sensordb and that's

where I got the statistics from so far. So
if you're interested in what a new phone

that you're about to buy can actually do.
Of course we don't give you any guarantee,

but you can check up or check out all the
data, all the phones. At least those that

are already in our database. And of
course, I'm happy if you contribute

statistics about the census in your phone
as well. So you might want to play with

this later. And then finally, the last
thing to finally conclude is some

information on how you can access the
sensors. Of course you can write your own

APP. I think here quite a few who can do
this. Just have a look if you can write an

App. Have a look at the API. They're not
too complicated. It's easy to access the

sensor data. If you're not interested in
designing your own app, but you want to

include sensor data in some other
projects, there are three ways you can use

Phyfox for this, which I want to
introduce, because that's something that's

one of the reasons I wanted to connect
here. Don't hesitate. Phyfox is free. You

can get for free on Google Play and on the
Appstore. And when I say it's free, I mean

it's really free. So it's open source. The
GPL and you can also get an afterwards we

assured of code running on your phone
is the code that you see. And we have

three versions how you can.. At least they
are categorized into three versions. How

you can access the sensor data. First
thing is you can implement something in

Phyfox yourself. So I've got this editor,
visual editor of all file format, which

allows you to take a sensor, place on
mathematics. So this is just adding stuff,

but you can apply a Fourier transform or
anything and then assign it to a graph.

Alternatively, and of course a bit more
powerful. You can have a look at our XML

format, which defines all the experiments.
So actually all experiments to see in

Phyfox are not hardcoded, but they are defined
in our own file format you can edit any of

them to your needs. And when you're done
you can transfer your data with the QR

code. Do not try to scan this QR code just
from your QR code app. You have to scan

it from within Phyfox and if you do,
you'll find a nice little experiment which

uses our file formats to implement a Turing
machine that's counting binary up to 256.

So this is the proof that all file format
actually is Turing complete. So you can do

a lot with it. Okay. I'm not suggesting
that you're trying to implement doom on it

or something like this because you won't
be able to. It's not efficient that way.

It's not designed to be Turing complete.
It just happens to be Turing complete. So

if you want to do something more, you can
connect to Phyfox via a network. You've

seen one example with the salad spinner.
When I said that there is a

a web server running on the App. You can use
this to access the data directly from your

preferred programing language. There's an
example where I'm using Python to read out

the sensor data and control a synthesizer.
So what's running on the web server is

basically a rest API. So yeah. Just visit
our website and learn how to do this. So

you can read out the sensor data of a
network and control your project with it.

An alternative to this is a new network
interface that we have, which is more on

this XML side or the design of our
experiment configurations, which is meant

to collect data from many users and not
life data. So we had this lecture. So this is

the new lecture hall, by the way. So we
had a lecture where every student got a

spring from us and there was supposed to
build a spring pendulum and we collected

the data from all students and the lecture
hall in realtime on the big screen to

determine the dependency of the frequency
from the mass of the pendulum. And another

example. Just a few days ago, we during
the winter solstice, we asked our

international users to point their phone
at the sun. So we get an angle for the

elevation of the sun and the azimuth from
the magnetometer with a compass. And this

way we could trace the path of the sun
across the earth from all the users. What

each black point with the line is a
contribution from a user. So, yeah, from

this we could, for example, determine the
tilted angle of the earth's axis. OK, so

just example, what you can do is this
network interface, as long as we're able

to set up some server to receive the data,
you can use this network interface. We're

still working on this network interface.
So far it can only do HTTP requests, get

or post. But we are also planning on
implementing Mqtt and other protocols like

this. And the third option is a Bluetooth
connection, which is mostly designed for

sensors. So if you want. If you have some
Bluetooth low energy sensor that you want

to read out, you can use Phyfox. So
there's an example of a Texas Instruments

sensor tech, which has a software which is
not designed for Phyfox. But our file

format is flexible enough to
simply tell Phyfox how to read all the

data and suddenly we've got the sensor
that can run independently from the phone.

And of course you can include your own
projects like this. So there is an example from

actually my institute, because originally I'm in
solid state physicist. So we're working a

lot with graphene and this is a
demonstrated we create that was an ESP 32.

So this is another version of an Arduino, or
Arduino compatibel. What we're doing here.

We're reading out a graphene Hallsensor
and so. It's all similar to the holecenter

of phone, but based on graphene and we can
get life measurements in Phyfox with this.

And so if you have an Arduino project with
which you want to.. from which you want

to send data that is plotted in Phyfox,
you can do it with a bluetooth low

energy interface. But if you have some
patients and maybe wait two more months,

we are working on Arduino library to make
this simpler. So this the entire code, you

would need to read out the analog input
from an Arduino and send it to Phyfox to

be plotted. OK, so this is working right
now. If you cannot wait, you can check it

out on our website. So this is already
available, although it's a work in

progress. The interface will change a bit
still. I would prefer if you want to

start right now, if you contact me so we
can get some feedback and maybe even

design the library also to your needs. So that
we get an idea. So with this, I'm about to

finish. So just a short summary what I'm
hoping I can trigger. Yeah. So if you were

mildly amused, mightily entertained by
this by this talk, check out our Web site

or check out our YouTube channel or
Twitter. We can get some more examples,

what we do with the sensors in the phone.
If you are a teacher, are teachers here?

Quite a few. That's great! And if you want
to use this in class or in a lecture,

check out our Web site phyfox.org. We've
got a database of experiments that you can

do: phyfox.org/experiments .That's then
actually about physics and less about the

hardware where we also demonstrate the
experiments and how they work. If you are

a teacher and has a specific project in
mind. Check out our editor to design your

own set up with which you can do something
specific for a very specific experiment.

phyfox.org/editor. Then if you are working
on arduino project and want to plot

something, you can visit Phyfox.org/arduino,
where you already can access

our library. Although it's not complete as
I said. So maybe wait a little bit or

contact me first. If you have a Bluetooth
low energy device that you want to use or

integrate. You can visit phyfox.org/ble.
If it's about a device that you did not

design yourself, you probably need some
background information about bluetooth low

energy. Should know what a GATT server is
and how characteristics and services, new

ideas and all this stuff and bluetooth
energy works. And it's good to get some

documentation or to be good as reverse
engineering, but in principle I haven't

seen many devices so far which could not
work with phyfox easily. Then if you want

to read the values for another project via
network, visit our website, the wiki on

our website. phyfox.org/wiki, where you
can get information about the rest API and

on your network interface. And finally,
something I would really love if you want

to contribute. If you can write some apps,
I mean you can use a lot of things. The iOS

app is written in swift. The Android
version is written in Java. Our webserver,

of course, has web development and Html in
JavaScript. So if you want to contribute

there. Visit our Web site at a
phyfox.org/source. And we would love to

see some help in development. With this I
finish my talk and I'm looking forward to

any exchange we will have later and any
questions. And I'm just thankful that it

was allowed to talk here and
get so much attention. Thank you.

Applause

Oh, by the way, since it is up there. One
bad news, unfortunately, I can only be

here today. So if you want to talk to me,
try to catch me today. You can also call

me. I actually brought a DECT phone,
but, uh, sorry, only today.

Herald: Oh, my God. So quickly, though, we
have questions now, 15 minutes, then 15

minutes, I think. And then afterwards, you
have to find him and catch him. Thank you,

Sebastian. Questions. Shoot. There is one.
Question: You mentioned aliasing affect

during.. Is it possible to change or modulate
the sampling frequency to actually find

out our frequency above the sampling frequency?
Sebastian: Yeah, that's that's a good

question. Not only because of the of the
alias affect, but also because some

projects also want to reduce the sampling
frequency. It's a little bit tricky

because on both APIs and both IOS and
Android, you cannot specify a target

frequency, you can only specify a
frequency that specific for certain use

case. So for example, you say I need the
accelerometer data, that's which at a rate

that's reasonable for UI changes or at a
rate that's reasonable for games. Right.

Right. That's as fast as possible. So if
you do it for UI, you get something like

let's say two three hertz. We heard something like
this or you doesn't waiting ages before

the screen rotates for games. It's 25 50
hertz something like this. So we can

control the game and fastest is the
data I've just plotted. And Phyfox always

request the fastest we can see and in
Phyfox we have a setting, we can limit the

frequency. Unfortunately, if your
frequency is not simply a multiple, no,

the other way around is. The frequency
given by the device, is not a multiple of the

frequency that you gave. It's not easy to
break it down to the target frequency. So

you usually see some odd cases where
Phyfox tells to group the sensor events

along this to get near this frequency. So
it might not work that well. And

especially if you're looking for the alias
effect. This might really mess up their

alias effect, so you might need to try a
little bit which frequency looks good to

do this. But of course in principle you
can average about multiple values in this

way or simply pick only every end value.
And this way we'd use the frequency. And

yeah, this can be done to our editor or to
the main screen. There's a plus button

with which you can simply expand which
already allows you to set this simple

frequency. Just keep in mind that you
cannot really always get to the

target frequency, right?
Herald: Right. There is another question.

No? Yes. Please.
Question: Hi. Thanks for the cool task.

It's a great app. I love using it in
school. I was wondering if those cool

animations how to sensor types of working
are available.

Sebastian: Sorry. The animation scene.
Yeah. I think I wonder how to do this

best. Before that, I was already thinking
about sharing the slides. Actually, my

talk is space it's just written in HTML in Javascript
it's not easy to control for everyone. That's

why I did not simply upload it. I would if
I would check later, if I can, upload the entire

talk in some way that makes sense either
on our website. I'm not sure if it makes

sense to upload it to the system of the
conference. Still, after the talk, I would

check it, but I am not... I want to share
the slides, but I probably need to add

some documentation on how to use them
because they are not Power point PDF or

Latex generated PDF.
It's handmade.

Herald: You can always cut them out of the
video getting streamed and La la la la la.

Yeah, right.
Question: Just a quick question of the the

axis of the phone. They're like like that
and that distorts us.

Sebastian: So it's not for most phones.
The X-axis is reading directlon. The

Y-axis is upwards along the screen and Z
access, Z-axis depending on your dialect is

perpendicular to the screen. I'd say in
most cases because officially the X-Axis

at least I think I've written this
documentation for Android is along the

natural reading direction of the device.
So if you've got a huge tablet which you

naturally would put in horizontal
alignment, not portrait mode, it might be

that the X-Axis is the long Axis. I have never
seen this myself, but I'm a little bit

careful to say that all these devices have
the same axis, but Z is definitely always

perpendicular to the screen and X and Y are
than the other ones and they are fixed and

usually the short side is the X-Axis.
Herald: Ok. There is one more question

there, please, sir. Take the microphone.
It's next to you. You got it off the

...
Question: Hi, you mentioned the necessity

of the magnetic sensor to to determine the
content orientation. Can you not use past

G.P.S. data and then integrate over the
gyroscope data to get the current

orientation?
Sebastian: Lauthing Mathematically, your

correct, problem is integrating sensor
data is not as simple. I'm often

surprised on what some software
can actually do. If you do it naively

right now I only have an example in mind
for the accelerometer cause it could also

say you can integrate the accelometer data
to get velocity. You can integrate the

velocity to get the displacement of the
phone of the location. If you do this,

we've got a very simple example in our
wiki. Very naiv even one without any

filtering, then just the noise means
that's if there's little arrow, you summit

up integrations, nothing else but
suming up in small steps. You get an

offset error in the velocity. If you
integrate this again, you get an error in

the location with which is growing with
the square of the time. So if you do this

for location and try it out with our naive
approach your phone is supposed to

be 100 meters upwards after about 10
seconds. If you do this for the gyroscope,

it's a little easier because you only
want integration. But still there will be

some drift. I'm not sure about all the
techniques the manufacturers imployed to

filter out any errors. I mean, obviously
the gyroscope is self calibrating

otherwise, it would be pointing in
different direction all the time. And on

some phones I've seen it jumping
when it recalibrates. But if you simply

integrate this, you will certainly get
drift, there's no way that you can get

a fixed position. What I think what they
probably do for most cases, they use the

gyroscope to immediate direct
rotation and then try to fuze it

in some way with the magnetometer
information to keep it fixed so that at

the end you're not pointing the wrong
direction. But the gyroscope itself, only

on its own, is unfortunately only giving
you the rotation rate, not the absolute

rotation in contrast to an actual
gyroscope. The big one that's rotating. So

it's at least not that easy.
That's all I can say.

Herald. Whow? What the bunch of
information, Sebastian? I really love the

.... There is someone else with a
question. I really love your replication.

Actually, it was really immediately fun
to, go.

Question: Thank you so much for a great
application. And my question is, just very

short. Can you also integrate external
sensors through Wi-Fi or is it only to be

early?
Sebastian: No. That's what I meant with

the network connection. Network usually
has Wi-Fi in this case, I'm not sure if it

would work on a conference like this into
the cable. So now you can get the data

through our REST API. Might not be the
fastest thing. Maybe we will add to our

network, our new network functionality,
something that will keep open apart and

push the data in there, so far the best
thing to go is with our rest API.

Question: I was just thinking about the
external sensor connection.

Sebastian: So external? Sorry, I was
thinking a different direction. Actually,

that's a good question. That reminds me of
that, that there's something I wanted to

add. You can use the REST API in theory to
push data in there, but that's only a

parameter in the Url. It's simple a Get/
push off a single value which doesn't get

get you far and which is quite
inefficient. However, within you network

interface you can do requests to other
devices so you can GET request and already

is able to respa Json packet as a
response us to interpret the adjacent

packet as a response. And that's where
adding Mqtt and stuff like this, this is

supposed to go in both directions. But
this is really new. So if you've got

something specific, try if it works or
contact me if it's not working, if you

need some help, if you find the bug. So but
it's supposed to work on your network

stuff. That's there in the configuration. So
the idea of the workflow of all this

connection with specific devices have
something set up like this. You create a

configuration for Phyfox, which in the end
is supplied to the QR code. For example,

the user scans the QR code. And this all
the information, how to communicate with

the device is already supplied. You can
also do this for Bluetooth. That the

device itself provides it to Phyphox.
But in the end it's these configurations

and for the new network interface, it can
also receive data from the network. But so

far only via HTTP.
Question: OK. Thank you.

Herald: I have maybe a last question if no
one else has. What's the next step? What

is your next goal? Because this is a
tremendous successful thing. And you see

the educational purposes. So that's
fantastic, actually, isn't it? It's not

only on university level if you're using
it, that's all around in Germany.

Sebastian: That's not in Germany. It's by
the way another thing you could

contribute. If you're speaking a language
that has been translated into Phyfox is

translated by volunteers and it's already
available, I think in 2010 and 2012, 2013,

14 languages, something around this. So
yeah, but next step I think will be using

the camera because that's another sensor,
broadly speaking, which we are not using

at all, which can do a lot, but we haven't
yet started on this. So lot to do in this

project.
Herald: Super. I'm looking forward to see

you next year then. Laughing, Applause
Sebastian Starks, thank you very much. An

honor and a pleasure to have you.

Postroll music

Subtitles created by c3subtitles.de
in the year 2020. Join, and help us!