0:00:00.000,0:00:03.730
[Translated by {Iikka}{Yli-Kuivila}[br](ITKST56 course assignment at JYU.FI)]
0:00:03.730,0:00:31.339
33C3 preroll music
0:00:31.339,0:00:35.200
Herald: Welcome back to the festival[br]stage, still day two according to my
0:00:35.200,0:00:40.970
clock, even though I have lost all sense[br]of time by now, I don't know. It's like a
0:00:40.970,0:00:45.440
kind of rush year with so many great[br]talks, and we'll have the next super
0:00:45.440,0:00:52.640
awesome talk held by Jiska, Gerbert and[br]Matthias. And those three lovely people
0:00:52.640,0:00:59.780
will be showing us something about VPNs.[br]So-called very pwnable networks and why
0:00:59.780,0:01:05.989
your VPN might not be as secure as you[br]think or have been led to believe. And
0:01:05.989,0:01:10.909
this is not only important for the people[br]that think like, haha, I'm behind seven
0:01:10.909,0:01:15.829
proxies. You can't catch me. But also,[br]maybe if your employer says, please use
0:01:15.829,0:01:21.179
this VPN client and there might be some[br]surprises waiting for you. And this is
0:01:21.179,0:01:26.350
again, a pre-recording, and afterwards I[br]can already see them here in our tools,
0:01:26.350,0:01:32.490
but afterwards we will do a Q&A session.[br]So if you have any questions, put them
0:01:32.490,0:01:40.859
into the IRC or on Twitter or Mastodon[br]with the hashtag #rc3cwtv. And our lovely
0:01:40.859,0:01:46.366
Signal Angel will collect them and we'll[br]see each other after the pre-recording.
0:01:46.366,0:01:48.459
Jiska: Everyone and welcome to the talk
0:01:48.459,0:01:56.170
Very Pwnable Network which is by Gerbert,[br]Matthias and me. So how did all start?
0:01:56.170,0:02:02.369
Well, I got a little bit paranoid last[br]year and I just thought it might be a good
0:02:02.369,0:02:08.510
idea to encrypt a lot because Wi-Fi, LTE,[br]TLS it might be intercepted, might be
0:02:08.510,0:02:14.420
decrypted. So we should encrypt like[br]everyone is watching and so I should use a
0:02:14.420,0:02:20.110
VPN on top. And then the next trust[br]assumption was that I connect to my
0:02:20.110,0:02:26.740
university's network every day, so I[br]should trust this one anyway. And if I
0:02:26.740,0:02:30.710
trust this network, then I could also use[br]this network and their professional VPN
0:02:30.710,0:02:36.060
service. I hear you laughing, I hear[br]laughing but so this was my idea because
0:02:36.060,0:02:42.570
then there is no additional thing that I[br]trust during my network activity. And
0:02:42.570,0:02:47.610
well, they had this nice AnyConnect line[br]that just works on all operating systems.
0:02:47.610,0:02:53.250
So it also sounds like a great product[br]name for a secure product like AnyConnect.
0:02:53.250,0:03:00.180
Yeah. And then I started using this on my[br]mobile devices, right? And then suddenly I
0:03:00.180,0:03:06.440
got crash logs and it looked like this. So[br]I mean, if you're paranoid, you obviously
0:03:06.440,0:03:11.840
look into your crash logs every two days[br]and one of these crash logs was this one.
0:03:11.840,0:03:17.170
It didn't really look nice the address[br]looks a bit strange, and you also have
0:03:17.170,0:03:21.850
some trace of the crashed thread. And if[br]you log this into IDA and do a little bit
0:03:21.850,0:03:25.540
of reverse engineering, you can also add[br]some function names in the AnyConnect
0:03:25.540,0:03:31.540
extension, the AC extension and the crash[br]is somewhere when it applies VPN config
0:03:31.540,0:03:37.160
somewhere in the - in the tunnel buffers.[br]I don't know. So - and then also, the
0:03:37.160,0:03:44.220
address looks strange because the address[br]is IP version four backwards and then a
0:03:44.220,0:03:52.020
few days later I got another crash and[br]this crash read like 68k New Line. And so
0:03:52.020,0:03:56.670
it really looked like some configuration[br]strings that were crashing at certain
0:03:56.670,0:04:02.110
addresses, so it really didn't look good.[br]But I also didn't have that much time. So
0:04:02.110,0:04:06.940
I just wrote Cisco and said, like, so do[br]you mean to serious like, is your client
0:04:06.940,0:04:13.180
really crashing that often? Because every[br]time I do my laundry, I have bad Internet
0:04:13.180,0:04:20.370
connectivity, so my any - my AnyConnect[br]client starts to crash at weird addresses.
0:04:20.370,0:04:26.889
And while this is like an issue for code[br]execution, the other issue is that if you
0:04:26.889,0:04:30.950
just jam a few packets over the air, then[br]an attacker might be able to disconnect
0:04:30.950,0:04:38.360
the victim from a VPN. So that is also an[br]issue. So because the AnyConnect client is
0:04:38.360,0:04:44.010
not properly included in the operating[br]system iOS, and that means that you would
0:04:44.010,0:04:48.130
just reconnect and send your messages in[br]plaintext over the network and the user
0:04:48.130,0:04:52.440
does not get any notification. It's just a[br]VPN symbol that has gone and all traffic
0:04:52.440,0:04:58.030
is sent in plaintext. And I wanted to[br]analyze this a little bit further and
0:04:58.030,0:05:01.590
there is a debugging option in the[br]AnyConnect client. But since it's in the
0:05:01.590,0:05:07.480
client that crashes, the logs are just[br]gone. If you have a crash and even worse
0:05:07.480,0:05:12.340
also the operating system crash logs in[br]iOS are missing if you have the debugging
0:05:12.340,0:05:19.180
option enabled. And well they answered:[br]"We cannot reproduce this". I mean,
0:05:19.180,0:05:23.520
obviously they didn't help me with my[br]laundry or anything so they could not
0:05:23.520,0:05:30.230
reproduce. And even worse, they ignored[br]the ticket for a long time until we said
0:05:30.230,0:05:37.100
we are going to present this at this[br]year's CCC congress. And then 10 days
0:05:37.100,0:05:40.520
before the talk they claimed that they had[br]fixed two out of the three crashes and
0:05:40.520,0:05:44.400
were asking if we could reproduce the[br]crashes or not. But just a day before they
0:05:44.400,0:05:51.510
said like they were not sure and so on. So[br]it was really a weird ticket there and I
0:05:51.510,0:05:55.040
have no idea if things are fixed or not,[br]because to reproduce, I obviously need
0:05:55.040,0:05:59.949
like tinfoil. I need like a couple of[br]smart phones with the AnyConnect client
0:05:59.949,0:06:04.250
installed, and then I need to walk around[br]and hope that the Internet connection
0:06:04.250,0:06:08.060
breaks and that it breaks in a way that[br]produces a crash. So this is just way too
0:06:08.060,0:06:17.030
random. I also told Apple and Apple just[br]said, Well, the issue is that the client
0:06:17.030,0:06:21.590
crashes so it's not really their[br]department, even though it's the network
0:06:21.590,0:06:28.840
extension that crashes. So it's not their[br]task here to notify the user. It's just
0:06:28.840,0:06:33.300
the VPN that is gone. And then you send[br]data in plaintext and this is all right.
0:06:33.300,0:06:41.170
This is the expected behavior. So I got a[br]bit annoyed and stopped using VPNs, but I
0:06:41.170,0:06:47.240
also had the idea to find students who[br]might look into this. And well, you know
0:06:47.240,0:06:52.369
what happens when you find very motivated[br]students, they find vulnerabilities. And
0:06:52.369,0:06:54.590
this is what this talk is going to be[br]about.
0:06:54.590,0:06:59.930
Gerbert: So we had a look at the Cisco[br]AnyConnect client for Linux and found some
0:06:59.930,0:07:04.460
interesting things. Shortly after[br]publishing our findings, several news
0:07:04.460,0:07:09.929
articles popped up. But what happened[br]between the crashes of Jiska and these
0:07:09.929,0:07:17.810
articles? VPN is interesting because it is[br]such an old research topic. However, the
0:07:17.810,0:07:23.110
importance of the technology was later[br]increased by the corona pandemic. Many
0:07:23.110,0:07:29.639
companies had to relocate to home offices[br]to meet the safety measures. VPN allows
0:07:29.639,0:07:36.240
users from outside to access internal[br]resources of the company or university. To
0:07:36.240,0:07:42.690
connect to a VPN server, additional client[br]software is usually required. Besides open
0:07:42.690,0:07:49.020
source products, such as openVPN, a lot[br]of closed source software is offered on
0:07:49.020,0:07:55.400
the market. Especially in the enterprise[br]sector. The users are forced to trust this
0:07:55.400,0:08:01.930
software, install a black box in their[br]system in order to connect to a network.
0:08:01.930,0:08:09.009
AnyConnect secure mobility client from[br]Cisco is an enterprise software solution.
0:08:09.009,0:08:13.650
AnyConnect can be classified as a remote[br]access client that allows end-users to
0:08:13.650,0:08:21.550
connect to a network that supports SSL VPN[br]and also IPsec. When using SSL VPN the
0:08:21.550,0:08:29.510
authentication and the establishment of a[br]VPN tunnel is carried out via an SSL or
0:08:29.510,0:08:35.650
TLS tunnel. The software acts as a fat[br]client that communicates with the VPN
0:08:35.650,0:08:42.329
server via HTTPS. As already mentioned the[br]source code for AnyConnect is not publicly
0:08:42.329,0:08:48.500
available. The application is distributed[br]with compiled binaries and libraries.
0:08:48.500,0:08:54.210
Although the application is documented, it[br]does not cover internal functionality.
0:08:54.210,0:08:59.160
Even the vulnerability disclosures, which[br]are published in public advisories, do not
0:08:59.160,0:09:04.630
go into much technical detail. Therefore[br]there is only limited knowledge about the
0:09:04.630,0:09:10.890
application and its internal behavior. We[br]have set ourselves the goal to examine
0:09:10.890,0:09:19.649
AnyConnect for Linux and iOS in a recent[br]version. The main functionality of
0:09:19.649,0:09:25.380
AnyConnect is the establishment of VPN[br]connections, but this is only the tip of
0:09:25.380,0:09:31.680
the iceberg. AnyConnect also connects[br]numerous other features, to name a few:
0:09:31.680,0:09:37.230
The distribution and execution of scripts[br]on host systems, automatically updating
0:09:37.230,0:09:43.470
the software without asking the user for[br]permission. Another feature is host scan:
0:09:43.470,0:09:49.310
it does not integrate into AnyConnect, but[br]it is considered as a standalone software.
0:09:49.310,0:09:54.180
It works together with the AnyConnect[br]infrastructure and makes it possible to
0:09:54.180,0:10:00.550
read out extensive system information of[br]the host and transmit them to VPN server.
0:10:00.550,0:10:08.870
The related work of AnyConnect is based on[br]Cisco's public advisories and blog entries
0:10:08.870,0:10:14.660
from certain security researchers.[br]Therefore, we decided to list and
0:10:14.660,0:10:21.470
categorize all vulnerabilities since 2011.[br]On this diagram, we see a list of
0:10:21.470,0:10:27.430
vulnerabilities per year, ordered by[br]severity. Most of the time reported
0:10:27.430,0:10:33.290
vulnerabilities are classified as medium,[br]but even critical vulnerabilities were
0:10:33.290,0:10:44.010
disclosed in 2011 and 2012. The increased[br]numbers in 2015 and 2016 caused numerous
0:10:44.010,0:10:51.380
vulnerabilities of libraries such as[br]OpenSSL. The vulnerabilities were then
0:10:51.380,0:10:57.420
divided into categories and illustrated in[br]the diagram. Cryptographic vulnerabilities
0:10:57.420,0:11:04.370
are the most common as AnyConnect uses[br]OpenSSL and vulnerabilities in OpenSSL are
0:11:04.370,0:11:10.050
also vulnerabilities in AnyConnect.[br]However, excluding the third-party
0:11:10.050,0:11:15.770
vulnerabilities, the category of most[br]vulnerabilities affecting AnyConnect would
0:11:15.770,0:11:21.510
be privilege escalations. Closely followed[br]by denial of service attacks, which are
0:11:21.510,0:11:26.110
often directed against the running[br]application in order to disrupt the VPN
0:11:26.110,0:11:33.110
connection. Overflow vulnerabilities or[br]version downgrades
0:11:33.110,0:11:37.920
These results gave us a little insight[br]about the flaws of the past.
0:11:37.920,0:11:43.279
Matthias: So before we will pass over to[br]the reversing parts, we want to give you a
0:11:43.279,0:11:50.540
quick experience report regarding Cisco's[br]licensing support. Cisco ASA is a typical
0:11:50.540,0:11:55.589
server endpoint for Cisco AnyConnect,[br]which is available as hardware appliance
0:11:55.589,0:12:03.430
and also as a virtual version running on,[br]for example, VMware. At the beginning of
0:12:03.430,0:12:08.089
our research, we tried to obtain an[br]official evaluation license for Cisco
0:12:08.089,0:12:14.270
ASAv, while never hiding the purpose of[br]wanting to use it for security research.
0:12:14.270,0:12:21.290
Therefore our naive approach was to just[br]write the Cisco licensing support. It was
0:12:21.290,0:12:25.880
more or less like: Hi, we are doing[br]security research on the Cisco AnyConnect
0:12:25.880,0:12:32.130
clients. Could you offer us free evalution[br]licenses for Cisco ASAv? Cisco replied:
0:12:32.130,0:12:38.220
Sure. Please let me know the amount of[br]licenses that you need. At that point, we
0:12:38.220,0:12:43.290
were a bit surprised because it just[br]seemed a bit too easy. But then we had
0:12:43.290,0:12:50.170
three 60-day licenses added to our[br]account. But as soon as we tried to
0:12:50.170,0:12:55.940
download the ASAv image for VirtualBox, we[br]got this error. Maybe some kind of error
0:12:55.940,0:13:00.930
on the license was not applied correctly,[br]because if you have the license for
0:13:00.930,0:13:06.209
products, you can of course download it,[br]right? Um yeah, but it seemed that our
0:13:06.209,0:13:11.090
approach was a little bit naive and we[br]underrated the complexity of enterprise
0:13:11.090,0:13:20.939
software licensing. So it was a bit[br]unsatisfying. But we had an idea. We asked
0:13:20.939,0:13:26.120
the data center of our university for help[br]as university was using Cisco AnyConnect
0:13:26.120,0:13:32.829
too. But we never got any response. We[br]still had some options, but at some point
0:13:32.829,0:13:38.540
we just gave up.[br]G: As there is no proper documentation for
0:13:38.540,0:13:44.230
AnyConnect yet, it was initially necessary[br]to understand the application and to
0:13:44.230,0:13:49.690
filter out its central components.[br]Therefore, we had no choice but to reverse
0:13:49.690,0:13:58.649
engineer the application. We analyzed the[br]application files and network traffic.
0:13:58.649,0:14:03.980
In order to better understand the applica-[br]tion we used standard tools like Ghidra
0:14:03.980,0:14:09.560
for static code analysis. The Ghidra is[br]able to decompile the source code of the
0:14:09.560,0:14:15.899
compiled binary or library. For dynamic[br]application analysis we used tools like
0:14:15.899,0:14:22.740
Frida, Burp and Wireshark. Frida can be[br]used to attach to running processes and to
0:14:22.740,0:14:28.470
understand the program flow. Burp was used[br]as a proxy to view and modify https
0:14:28.470,0:14:35.899
messages, and Wireshark is used to capture[br]and decrypt message traffic. Let's take a
0:14:35.899,0:14:41.390
closer look at the binaries of all the[br]files first. AnyConnect comes with a large
0:14:41.390,0:14:45.890
number of binaries. While reversing we[br]could identify three central parts of the
0:14:45.890,0:14:52.160
application: vpnui is a binary a user[br]interacts with. It offers the graphical
0:14:52.160,0:14:58.570
user interface, where the user can make[br]simple settings or initiate a VPN
0:14:58.570,0:15:05.339
connection. The second binary is[br]vpnagentd. It runs as a daemon in the
0:15:05.339,0:15:11.670
background at all times, even when no VPN[br]connection is open. The special thing
0:15:11.670,0:15:17.680
about vpnagentd is that it runs as a root[br]process and always listens on a static
0:15:17.680,0:15:25.130
port. Its purpose is to set up the VPN[br]tunnel and the network configuration of
0:15:25.130,0:15:32.899
the host system. This includes setting up[br]routes or DNS servers. The third and last
0:15:32.899,0:15:38.040
binary is the vpndownloader. As the name[br]hints, the purpose of the binary is to
0:15:38.040,0:15:43.570
download additional files when[br]establishing a VPN connection. This
0:15:43.570,0:15:51.020
includes VPN profiles, help files and[br]scripts. The binary exchanges data with
0:15:51.020,0:15:59.819
each other via inter-process communication[br]or short IPC. The IPC takes place via TCP
0:15:59.819,0:16:05.840
sockets. The binary data format for - that[br]Cisco has defined is used to exchange the
0:16:05.840,0:16:15.670
messages on the TCP sockets. In addition[br]to the binaries, AnyConnect also contains
0:16:15.670,0:16:20.960
numerous libraries. Many of them are ports[br]of existing open source libraries like
0:16:20.960,0:16:29.620
OpenSSL. The most important libraries are[br]shown on this slide: libvpnapi.so contains
0:16:29.620,0:16:35.670
interfaces and functions for the backend[br]logic of user interfaces. The goal of the
0:16:35.670,0:16:41.089
library is that it com - that companies[br]can create their own VPN applications
0:16:41.089,0:16:47.240
using the AnyConnect infrastructure. It is[br]the only library for which documentation
0:16:47.240,0:16:54.420
is actually provided by Cisco.[br]Libvpncommoncrypt serves as a wrapper for
0:16:54.420,0:17:01.430
OpenSSL and NSS libraries. NSS is similar[br]to OpenSSL and is used by browsers like
0:17:01.430,0:17:07.860
Mozilla Firefox to enable SSL and TLS[br]connections. It also provides its own
0:17:07.860,0:17:16.709
certificate store. Libvpncommon is another[br]central library used by all binaries. It
0:17:16.709,0:17:23.189
provides classes and functions for the IPC[br]logic. It can be used to create, send or
0:17:23.189,0:17:30.290
validate lPC messages. The next library,[br]libvpnagentutilities, contains classes and
0:17:30.290,0:17:36.700
functions that handle critical operations,[br]such as host network settings. This
0:17:36.700,0:17:45.150
library is only used by vpnagentd. Besides[br]the binaries and libraries there are a
0:17:45.150,0:17:51.720
variety of other relevant files that we[br]have taken a closer look at. AnyConnect
0:17:51.720,0:17:59.420
offers an AnyConnect local policy XML[br]file. This file regulates various security
0:17:59.420,0:18:04.420
configurations. For instance, it can be[br]used to specify that no further files may
0:18:04.420,0:18:11.740
be downloaded or that version updates may[br]be carried out by a VPN server. In its
0:18:11.740,0:18:15.840
default configuration it is very[br]permissive so that almost everything is
0:18:15.840,0:18:20.911
allowed. The file is not overwritten by[br]updates and cannot be modified by VPN
0:18:20.911,0:18:30.830
servers. The VPN profile is also in XML[br]format: it contains further settings. The
0:18:30.830,0:18:36.960
green highlighted line shows an[br]EnableScripting-tag with a boolean value
0:18:36.960,0:18:43.820
of false. Indicating that scripts should[br]not be executed by the host system.
0:18:43.820,0:18:50.200
Profile files are distributed by a VPN[br]server. And are overwritten the next time
0:18:50.200,0:19:00.450
a user connects and changes them. The last[br]file is VPNManifest.dat which has a binary
0:19:00.450,0:19:05.940
data format that contains the version[br]number of AnyConnect. This file is used to
0:19:05.940,0:19:11.800
check the installed version of AnyConnect[br]before an version update. In addition to
0:19:11.800,0:19:19.290
all these files, the message traffic also[br]plays a central role. The establishment of
0:19:19.290,0:19:24.870
a VPN connection is structured in three[br]phases. Phase one is the authentication.
0:19:24.870,0:19:33.750
The user enters the IP or domain of a VPN[br]server in vpnui. The target server is then
0:19:33.750,0:19:41.490
sent via IPC message to vpnagentd. As a[br]response vpnui receives various system
0:19:41.490,0:19:48.679
information back. This includes the[br]operating system or a whole study.
0:19:48.679,0:19:53.790
Afterwards this information and the[br]credentials are sent to the VPN server
0:19:53.790,0:20:04.160
with HTTPS. The ASA returns server[br]parameters in a HTTPS response. Let's take
0:20:04.160,0:20:10.150
a closer look at the request and the[br]response. On the left side, you can see
0:20:10.150,0:20:15.840
the request. It is an ordinary post[br]request with an XML in which the
0:20:15.840,0:20:22.220
credentials are transferred. The[br]credentials are marked in green.
0:20:22.220,0:20:28.130
On the right side we see the response. The[br]response contains the session token, which
0:20:28.130,0:20:33.420
is also marked in green. In addition, the[br]response contains the URLs to all
0:20:33.420,0:20:40.210
downloadable files and the hashes. The[br]orange marked string is one of the
0:20:40.210,0:20:47.580
downloadable files. The download phase is[br]the second phase of the VPN connection set
0:20:47.580,0:20:55.620
up. First vpnui executes the vpndownloader[br]binary. Then the server parameters from
0:20:55.620,0:21:03.144
the previous HTTPS response are[br]transferred to the vpndownloader via IPC.
0:21:03.144,0:21:09.620
The URLs are extracted from the IPC[br]message and when the files are downloaded
0:21:09.620,0:21:18.310
to a temporary directory via HTTPS. The[br]downloader process informs the vpnagentd
0:21:18.310,0:21:24.320
via IPC to move the files to the[br]application directory. In the third and
0:21:24.320,0:21:31.010
final phase of VPN connection set up,[br]vpnui sends an IPC message to vpnagentd
0:21:31.010,0:21:36.850
with the request to establish a VPN[br]tunnel. Subsequently, the exchange of
0:21:36.850,0:21:44.145
tunnel parameters takes place via HTTPS.[br]After the parameters have been set by
0:21:44.145,0:21:50.640
vpnagentd the VPN session continues. Let's[br]take a closer look at the tunnel
0:21:50.640,0:21:57.970
parameters. On the left side, we can see[br]the request. In the first line you can
0:21:57.970,0:22:03.990
observe the HTTP connect method. Usually[br]this method is used to proxies to forward
0:22:03.990,0:22:10.080
the request to the target server. Within[br]the request, the session token is
0:22:10.080,0:22:15.530
specified in the cookie header. This is[br]the same session token we received in the
0:22:15.530,0:22:22.330
authentication phase. The different tunnel[br]parameters transmitted in separate HTTP
0:22:22.330,0:22:30.160
headers. The part marked in red represents[br]the local IP address of the host. On the
0:22:30.160,0:22:35.660
right, you can see a response to the[br]request. For example, the X-CSTP-Address
0:22:35.660,0:22:43.470
header contains the IP address that the[br]host should apply on its tunnel interface.
0:22:43.470,0:22:50.060
In the part marked in red we now also see[br]the DNS server for the VPN
0:22:50.060,0:22:56.090
connection. In addition, the address[br]ranges that should be routed via the VPN
0:22:56.090,0:23:04.402
server as specified in the X-CSTP-Split-[br]Include header. Now that we have a general
0:23:04.402,0:23:09.569
understanding of the application, let's[br]move on to the vulnerability research. We
0:23:09.569,0:23:15.919
have performed an design analysis for[br]AnyConnect in which we looked at the IPC
0:23:15.919,0:23:21.729
messages in more detail. We need to define[br]certain security assumptions and an
0:23:21.729,0:23:26.400
attacker model before we search for[br]vulnerabilities. This slide shows several
0:23:26.400,0:23:33.020
of our assumptions. Cryptographic[br]algorithms within the application are
0:23:33.020,0:23:39.460
considered secure and cannot be broken in[br]exponential time. An attacker cannot read
0:23:39.460,0:23:44.679
or modify messages regardless of their[br]position, and we assume that the VPN
0:23:44.679,0:23:51.170
server does not pursue any malicious[br]intent and only sends valid messages that
0:23:51.170,0:23:56.780
are protocol compliant. We assume a local[br]attacker who is already able to execute
0:23:56.780,0:24:01.460
commands on the system and the attackers[br]goal is to compromise the confidentiality,
0:24:01.460,0:24:08.340
integrity and availability of the system[br]or application. Privilege escalation
0:24:08.340,0:24:13.830
vulnerabilities are also covered since[br]they allow an attacker to compromise these
0:24:13.830,0:24:20.550
three security objectives. Cisco decided[br]to include an auto update feature in the
0:24:20.550,0:24:27.929
application. AnyConnect is able to receive[br]AnyConnect updates through a VPN server
0:24:27.929,0:24:32.470
without any user interaction. In its[br]default configuration AnyConnect can be
0:24:32.470,0:24:38.740
updated by a VPN server that offers a[br]newer version. From a security
0:24:38.740,0:24:44.950
researcher's perspective auto update[br]sounds promising, right? So let's take a
0:24:44.950,0:24:49.510
closer look at the auto update feature.[br]First, the vpndownloader downloads an
0:24:49.510,0:24:57.010
executable installer and the shell script[br]called vpndownloader.sh. Then
0:24:57.010,0:25:03.630
vpndownloader.sh is executed. The Shell[br]script contains an archive and unpacks
0:25:03.630,0:25:09.380
itself to extract a new version of the[br]vpndownloader. An IPC message is then sent
0:25:09.380,0:25:15.740
to the vpnagentd asking it to start the[br]installer. The vpnagentd does not start
0:25:15.740,0:25:21.429
the installer directly. Instead the[br]vpnagentd calls the vpndownloader with
0:25:21.429,0:25:28.059
root privileges, which in turn calls the[br]installer. Before executing the installer
0:25:28.059,0:25:35.100
vpndownloader verifies and validates it.[br]We got an idea: Is it possible to install
0:25:35.100,0:25:40.909
an outdated version through forged IPC[br]messages? As shown in the picture the
0:25:40.909,0:25:47.780
attacker needs an old signed installer and[br]sends the IPC message to vpnagentd asking
0:25:47.780,0:25:53.679
to execute the old installer. The[br]vpnagentd calls the vpndownloader as
0:25:53.679,0:25:58.710
usual, which in turn calls the attacker's[br]installer. There is no check whether the
0:25:58.710,0:26:05.289
installer is more recent than the[br]installed version. This makes the version
0:26:05.289,0:26:10.720
downgrade therefore possible. The[br]advantage of downgrading to an outdated
0:26:10.720,0:26:15.650
version is that an attacker could force[br]the installation of a version which
0:26:15.650,0:26:21.179
suffers from security vulnerabilities and[br]the attacker could then exploit these
0:26:21.179,0:26:28.520
vulnerabilities. We reported the[br]vulnerability to Cisco's product incident
0:26:28.520,0:26:34.020
response team and it was fixed at the end[br]of September. The vulnerability only
0:26:34.020,0:26:42.390
received a CVSS score of 3.1 and was[br]therefore rated with a low severity. The
0:26:42.390,0:26:47.959
vulnerability was only exploitable in the[br]Linux version. Windows and Mac versions
0:26:47.959,0:26:54.600
were already secured against such an[br]attack. Another functionality we had - we
0:26:54.600,0:26:59.720
have looked into is the deployment and[br]execution of scripts. They call it "Bring
0:26:59.720,0:27:05.730
your own script". This functionality is[br]intended to deploy very helpful scripts to
0:27:05.730,0:27:11.350
staff computers. In order for a script to[br]be executed, it must meet two criterias.
0:27:11.350,0:27:16.880
First, it must be located in the script[br]folder and begin with OnConnect or
0:27:16.880,0:27:24.150
OnDisconnect as a file name. Second, the[br]EnableScripting tag in profile which is
0:27:24.150,0:27:31.760
sent by the server must be set to true.[br]Depending on the file name, scripts are
0:27:31.760,0:27:37.002
triggered after VPN connection is[br]established and terminated. As VPN server
0:27:37.002,0:27:42.980
can distribute profiles in which the[br]execution of scripts is enabled and also
0:27:42.980,0:27:48.289
distributes the scripts: these two in[br]combination allow VPN server to gain
0:27:48.289,0:27:54.410
remote code execution on the connecting[br]clients. This functionality poses a major
0:27:54.410,0:27:59.299
problem because humans often need to trust[br]the university's VPN servers and have no
0:27:59.299,0:28:05.539
other choice. But let's take a closer look[br]at the distribution of the scripts. Here
0:28:05.539,0:28:11.850
we see the classic procedure of a script[br]distribution. In the download phase,
0:28:11.850,0:28:19.171
vpndownloader downloads an OnConnect or an[br]OnDisconnect script. Then vpndownloader
0:28:19.171,0:28:25.419
asks with IPC message to move the[br]downloaded script to the script folder.
0:28:25.419,0:28:32.570
The vpnagentd process calls the[br]vpndownloader, which then moves the file.
0:28:32.570,0:28:36.670
We systematically examined the IPC[br]messages and found a vertical privilege
0:28:36.670,0:28:43.100
escalation. An attacker is able to send[br]the same message to the vpnagentd process.
0:28:43.100,0:28:48.350
Any of the attacker's scripts can be moved[br]into the script directory. If there is
0:28:48.350,0:28:52.970
already a script in the script folder, it[br]is simply overwritten. If the attacker
0:28:52.970,0:28:58.620
moves On, an OnDisconnect script while[br]the user is already - has a VPN connection
0:28:58.620,0:29:04.610
open, the script is executed with user[br]privileges. When the VPN connection is
0:29:04.610,0:29:13.220
closed, first unprivileged user can obtain[br]code execution context of another user.
0:29:13.220,0:29:18.450
What bothered us about our attack was that[br]it is - was tied to conditions. One of the
0:29:18.450,0:29:24.040
conditions was that the EnableScripting[br]tag must have the boolean value "True". We
0:29:24.040,0:29:29.020
considered other attack scenarios and came[br]up with the idea of distributing a profile
0:29:29.020,0:29:34.620
ourselves. So we check for the tag again,[br]but create not only a script, but also a
0:29:34.620,0:29:41.210
VPN profile that allows scripting. The[br]attack works as follows: while a local
0:29:41.210,0:29:45.580
user has a VPN session active, another[br]user on the system creates a malicious
0:29:45.580,0:29:50.539
script and a new profile. The new profile[br]contains the EnableScripting tag set to
0:29:50.539,0:29:57.059
"True". The attacker then sends an IPC[br]message to the vpnagentd requesting to
0:29:57.059,0:30:03.029
copy the script to the script directory.[br]The vpndownloader is then started with
0:30:03.029,0:30:10.990
root permissions to perform the copy[br]operation. Also, the attacker can send an
0:30:10.990,0:30:17.039
additional IPC message to vpnagentd[br]requesting to overwrite the
0:30:17.039,0:30:24.809
existing profile with a malicious profile.[br]Although the profile is overwritten the
0:30:24.809,0:30:29.109
settings of a new profile are not applied[br]yet, because the old profile is still
0:30:29.109,0:30:34.950
active. However, we were able to determine[br]that the new profile is loaded when
0:30:34.950,0:30:41.503
there's a reconnect on the VPN session.[br]Reconnects are actually quite common in
0:30:41.503,0:30:46.329
the AnyConnect. If reconnect happens the[br]new profile is loaded and applied. In our
0:30:46.329,0:30:51.230
case, it enables the scripting feature.[br]After teardown of a VPN connection the
0:30:51.230,0:30:56.490
malicious OnDisconnect script of the[br]attacker is executed with the privileges
0:30:56.490,0:31:03.010
of the user running the VPN client. Both[br]problems were reported to Cisco, but could
0:31:03.010,0:31:07.580
not be fixed by the disclosure date.[br]Although we extended the disclosure
0:31:07.580,0:31:12.020
deadline. As of today the vulnerability is[br]still present with the default
0:31:12.020,0:31:18.029
configuration of AnyConnect. Cisco[br]published this vulnerability with a CVSS
0:31:18.029,0:31:25.910
of 7-1, 7.1, which is considered as a high[br]severity. Because it was published on the
0:31:25.910,0:31:32.340
4th of November 2020 without a fix, the[br]vulnerability got major attention in many
0:31:32.340,0:31:42.410
news sites. Various sites reported the[br]vulnerability. The quality of reports
0:31:42.410,0:31:47.410
varied. Some of the articles contained[br]incorrect information, for example, it was
0:31:47.410,0:31:52.820
stated that an exploit was already in[br]circulation. It is not accurate since we
0:31:52.820,0:31:57.990
are the only ones who have a working[br]exploit and have not published it yet. We
0:31:57.990,0:32:03.190
could not find any other exploit on the[br]Internet either. I think that the way of
0:32:03.190,0:32:09.539
reporting - this way of reporting has to[br]be reconsidered because it has caused a so
0:32:09.539,0:32:15.450
wrong assessment of vulnerability. We were[br]even contacted by some incident response
0:32:15.450,0:32:21.370
teams that were worried about their[br]infrastructure. All vulnerabilities found
0:32:21.370,0:32:25.130
and reported are listed in the table[br]below. The three vulnerabilities were
0:32:25.130,0:32:30.350
found by design analysis. Only one of the[br]vulnerabilities is fixed, according to
0:32:30.350,0:32:37.980
Cisco. Especially the Bring Your Own[br]Script vulnerabilities have already been
0:32:37.980,0:32:42.760
published, although no fix is available[br]for them, a workaround has been declared
0:32:42.760,0:32:49.860
to fix the vulnerabilities. By modifying[br]the local policy file the download phase
0:32:49.860,0:32:54.450
can be skipped completely. Since the[br]latest update it's also possible to
0:32:54.450,0:32:59.679
prohibit the download and deployment of[br]scripts on a modular basis.
0:32:59.679,0:33:05.309
M: And what about mobile platforms? Do our[br]discovered vulnerabilities also apply
0:33:05.309,0:33:11.720
here? Let's make it quick. No. Mobile[br]platforms are lacking many features
0:33:11.720,0:33:15.740
compared to the Linux, Windows and macOS[br]versions. You're, of course, able to
0:33:15.740,0:33:20.730
establish TLS and IPsec connections, just[br]like with all the other clients. But
0:33:20.730,0:33:24.649
because features like the deployment of[br]custom scripts or auto update is missing,
0:33:24.649,0:33:30.010
there's no way of using these exploits on[br]mobile platforms. We are currently having
0:33:30.010,0:33:35.000
a look into the iOS implementation of[br]AnyConnect and therefore want to give you
0:33:35.000,0:33:41.430
a quick and high level overview into the[br]architecture. As we are dealing with Apple
0:33:41.430,0:33:45.570
here, serious stuff like the[br]implementation of the VPN is a bit
0:33:45.570,0:33:51.929
different compared to, for example, the[br]Linux client. If you want to mess with
0:33:51.929,0:33:56.399
notifications, add sharing buttons or[br]create a widget on the homescreen, you
0:33:56.399,0:34:01.740
have to use an app extension. Equally for[br]VPN functionality, you have to use the
0:34:01.740,0:34:08.429
network extension framework. The network[br]extensions contain providers and features
0:34:08.429,0:34:14.099
for all kind of network related operations[br]like for content filtering, DNS, Wi-Fi and
0:34:14.099,0:34:18.940
more. If you want to build your own VPN[br]app, you have to choose between the
0:34:18.940,0:34:25.839
Personal VPN, the Packet Tunnel Provider[br]and the App Proxy Provider. In our case
0:34:25.839,0:34:30.079
the AnyConnect on iOS implements the[br]Packet Tunnel Provider as they are using
0:34:30.079,0:34:38.279
their own packet oriented protocol. Here[br]you can see the contents of the AnyConnect
0:34:38.279,0:34:43.469
app package, which is basically a zip file[br]containing all the executables and other
0:34:43.469,0:34:49.819
assets like images, etc.. The main[br]executable is just called AnyConnect. The
0:34:49.819,0:34:56.209
network extension is implemented in the[br]ACExtension binary. Beside these, there
0:34:56.209,0:35:01.799
are also several other app extension[br]implementations for the iOS sharing and
0:35:01.799,0:35:08.910
Siri functionality. So what happens if you[br]pressed the connect slider? After you hit
0:35:08.910,0:35:13.329
the slider, the network extension is[br]started and negotiation of the VPN session
0:35:13.329,0:35:19.059
begins. After a section of network[br]information like IP addresses, subnet
0:35:19.059,0:35:23.130
mask, routes, DNS, MTU and more they are[br]passed to the iOS system to complete the
0:35:23.130,0:35:29.390
negotiation. In the end, you have a new[br]and hopefully functional tunnel interface
0:35:29.390,0:35:36.930
called utun. So new traffic from apps pass[br]through the network stack until it arrives
0:35:36.930,0:35:41.619
at the tunnel interface. It can then be[br]handled by the network extensions' Packet
0:35:41.619,0:35:48.269
Tunnel Provider. Every time a packet[br]arrives on a tunnel interface, it is read
0:35:48.269,0:35:54.191
by the network extension and encapsulated[br]with the tunneling protocol. Every time a
0:35:54.191,0:35:59.020
packet arrives on the tunnel interface, it[br]is read by the network extension and
0:35:59.020,0:36:04.660
encapsulated with the tunneling protocol.[br]Upon arrival on the VPN server, the packet
0:36:04.660,0:36:10.150
is decapsulated and sent to its final[br]destination. Similar the replies then
0:36:10.150,0:36:14.690
encapsulate sent to the client, which then[br]decapsulates the packet and injects it
0:36:14.690,0:36:20.499
back to the network stack. So that's[br]should be it for the iOS clients, just to
0:36:20.499,0:36:23.829
give you a quick overview and highlight[br]some key differences between the
0:36:23.829,0:36:30.930
architectures. We are still investigating[br]both Linux and iOS platforms, but until
0:36:30.930,0:36:35.700
now, we can summarize our findings as[br]follows: AnyConnect in general is a huge
0:36:35.700,0:36:42.530
application with lots of code and also[br]lots of unused library related code. We
0:36:42.530,0:36:48.510
discovered three vulnerabilities through[br]design analysis. Vpnagentd runs with root
0:36:48.510,0:36:53.529
permissions and receives commands or[br]operations from unprivileged processes via
0:36:53.529,0:36:59.549
unauthenticated IPC messages, which is[br]generally a bit risky. Not all security
0:36:59.549,0:37:03.999
mechanisms and patches are actually[br]included on all platforms. For example,
0:37:03.999,0:37:08.619
the downgrade was only possible on Linux[br]and already patched on Windows and macOS
0:37:08.619,0:37:14.369
according to Cisco. The downgrade[br]vulnerability is not possible on mobile,
0:37:14.369,0:37:19.430
as auto update feature is limited to[br]Linux, Windows and macOS. Similar, the
0:37:19.430,0:37:24.119
Bring Your Own Script does also not apply[br]in mobile, as deployment of OnConnect and
0:37:24.119,0:37:29.180
OnDisconnect script is also limited to[br]Linux, Windows and macOS. Regarding the
0:37:29.180,0:37:34.009
local policy file on Linux, our idea would[br]be to make it as restrictive as possible
0:37:34.009,0:37:39.219
and require some kind of opt-in for[br]scripting functionality. As this would
0:37:39.219,0:37:44.140
prevent many attacks but of course, it[br]would also impact the usability a bit.
0:37:44.140,0:37:48.779
Despite the app being available since many[br]years, we show that there are still many
0:37:48.779,0:37:54.279
bugs to find. The introduction of bug[br]bounties would be a great option to
0:37:54.279,0:38:00.859
motivate more security researchers to[br]check the application for vulnerabilities.
0:38:00.859,0:38:06.799
The use of VPN promises security and[br]privacy for users, however closed source
0:38:06.799,0:38:11.049
software opens new attack vectors on a[br]system as our research on AnyConnect
0:38:11.049,0:38:16.559
shows. We hope that more research will be[br]done on clients in the future and that our
0:38:16.559,0:38:24.460
work will pave the way for this. So that[br]was it. Thank you very much for your
0:38:24.460,0:38:29.610
attention and if you have any questions,[br]feel free to ask.
0:38:30.700,0:38:35.599
H: Well, welcome back. So that was the[br]pre-recording of the super interesting
0:38:35.599,0:38:41.239
talk about Very Pwnable Networks. I'm sure[br]you've seen how pwnable they really are
0:38:41.239,0:38:48.630
and luckily, we now have Jiska and Gerbert[br]and Matthias with us today through the
0:38:48.630,0:38:56.499
magic of the Internet. And if you haven't[br]written out your question yet, please do
0:38:56.499,0:39:05.869
so now so we can still answer them. Either[br]by going to the IRC channel rc3-cwtv on
0:39:05.869,0:39:12.219
the hackened network or just posting a[br]tweet or a toot with your favorite social
0:39:12.219,0:39:19.490
media network of choice containing the[br]hashtag #rc3cwtv this time without any
0:39:19.490,0:39:27.629
dash, very important. So our Signal Angel[br]has collected some questions for us that I
0:39:27.629,0:39:33.280
am now going to be torturing the three[br]people here with. So let's see what you
0:39:33.280,0:39:41.799
will say to those. Oh, but I think pretty,[br]pretty tame, huh? So first question: is
0:39:41.799,0:39:45.839
there any page or wiki where this[br]information can be found?
0:39:45.839,0:39:53.150
G: At the moment, it is not published yet.[br]I think we will publish - publish it in
0:39:53.150,0:39:58.952
near future on the GitHub or some similar[br]platform.
0:39:58.952,0:40:02.940
H: Is there any way that people can find[br]this link then when you eventually will
0:40:02.940,0:40:07.630
publish it in the future?[br]G: I think Jiska can say something to this
0:40:07.630,0:40:10.579
J: Yeah so SEEMOO has a GitHub page and
0:40:10.579,0:40:16.249
also a Twitter account, so it will be[br]published. But I mean, there's still a few
0:40:16.249,0:40:21.680
things that we didn't like - that are not[br]public yet. So yeah, and then we would
0:40:21.680,0:40:27.420
just make one release not like this CVE[br]and that CVE, but like all at once.
0:40:27.420,0:40:31.519
H: Yeah. Make - makes more of an impact[br]this way and also better, better
0:40:31.519,0:40:38.160
disclosure. I like that. And the next[br]question is: will this VPN event only be
0:40:38.160,0:40:46.420
about Cisco? So yes, you've only looked at[br]Cisco, right? In this case. Um, maybe you
0:40:46.420,0:40:50.729
can tell us something if maybe you have[br]looked at other VPN vendors as to
0:40:50.729,0:40:55.589
something that other VPN vendors might[br]have an issue with as well? Maybe.
0:40:55.589,0:41:01.670
M: Yeah, we've done this research as part[br]of a master's thesis, and therefore it's
0:41:01.670,0:41:10.059
only about Cisco AnyConnect and yeah, we[br]did not have the time to - or yeah. Yeah.
0:41:10.059,0:41:14.408
To look at other VPN services, just for[br]the AnyConnect.
0:41:14.408,0:41:19.630
H: Yeah, the typical Master's view writing[br]it hyped up on coffee for the last few
0:41:19.630,0:41:25.190
weeks before the deadline. Yeah, I can[br]imagine that you focused on Cisco there.
0:41:25.190,0:41:30.630
Uum, another very good question: Are these[br]vulnerabilities also present in other - in
0:41:30.630,0:41:35.039
other AnyConnect-like clients like the[br]ones integrated into NetworkManager in
0:41:35.039,0:41:40.309
Linux? I think they're talking especially[br]about OpenConnect. Yeah, that's just
0:41:40.309,0:41:45.749
coming in. Umm, we talked about this[br]before a little bit so you probably have
0:41:45.749,0:41:50.210
something to say about that.[br]G: As far as we know there's - in
0:41:50.210,0:41:56.999
OpenConnect there is no scripting -[br]scripting feature enabled or integrated.
0:41:56.999,0:42:03.630
So we would say this kind of attacks are[br]not yet possible, but uh -
0:42:03.630,0:42:10.559
M: That would be possible if someone is[br]able to check this and, yeah, have a look
0:42:10.559,0:42:15.680
into OpenConnect too, yeah.[br]H: Hmm, yeah, not yet known to be
0:42:15.680,0:42:24.059
possible. Let's - let's say it like that.[br]You never know. Any other questions from
0:42:24.059,0:42:31.439
chat, from Twitter or Mastodon? Now's the[br]time. Give out your questions. Otherwise,
0:42:31.439,0:42:40.480
this would have been a very short Q&A. So[br]if you go to the rc3-cwtv channel on the
0:42:40.480,0:42:49.319
hackened IRC network or post a tweet or a[br]toot with the hashtag #rc3cwtv this time
0:42:49.319,0:42:56.150
without any dash. Umm, do it now, and[br]hopefully I will still catch this
0:42:56.150,0:43:00.890
through the magic of the Signal Angel that[br]is collecting all this information for me.
0:43:00.890,0:43:05.549
And yeah, maybe anything that you want to[br]add or any other new topics that you're
0:43:05.549,0:43:09.339
working on, something that might be[br]upcoming. Maybe we can get a sneak
0:43:09.339,0:43:13.589
preview. I'm sure you're still continuing[br]the research there - and
0:43:13.589,0:43:15.799
J: Sorry I needed to unmute myself. Not
0:43:15.799,0:43:20.249
spoilering yet but I mean, the issue like[br]just looking into one VPN client it's
0:43:20.249,0:43:25.838
also, if it is proprietary, then just[br]reversing is a lot, a lot, a lot of work.
0:43:25.838,0:43:32.630
I mean, I can tell you a story about like,[br]looking into binaries for months, and so
0:43:32.630,0:43:37.819
it doesn't really scale to look into like[br]10 different clients, at least if you want
0:43:37.819,0:43:44.479
to have meaningful findings.[br]H: Yeah, but like you don't have any any
0:43:44.479,0:43:51.349
other plans to, you know, look at any[br]other clients that get any requests or any
0:43:51.349,0:43:56.609
triggers that would point you there. Maybe[br]having a new phone and a new client and
0:43:56.609,0:44:02.309
noticing new strange things? No? - Okay.[br]G: Not yet.
0:44:02.309,0:44:06.010
J: Yeah.[br]G: But, but yes, Matthias is still working
0:44:06.010,0:44:10.930
on it. I think he will find something in[br]the future. M: hopefully.
0:44:10.930,0:44:21.039
H: Yeah. Yeah. I guess that's pretty much[br]all the last questions, except one: There
0:44:21.039,0:44:26.339
are two shadows behind you Jiska, is that[br]the shadow cabinet?
0:44:26.339,0:44:29.629
J: No, no, no. I just have multiple lamps[br]here. So -
0:44:29.629,0:44:31.476
H: Yeah, yeah, it's funny everyone <??>
0:44:31.476,0:44:34.509
J: At least it's only showing my shadow[br]duplicate. Not - not me.
0:44:34.509,0:44:39.739
H: Yeah. But we're quite lucky that at[br]least in this talk, the connection seems
0:44:39.739,0:44:46.089
to be working fine. We've been having some[br]issues here. Yeah, but that's great. If
0:44:46.089,0:44:51.280
you have any other questions, I think[br]we'll wrap it up here not to keep you
0:44:51.280,0:44:57.420
around much longer. I'm sure you're eager[br]to jump around the rc3 world, and if you
0:44:57.420,0:45:03.469
have anything else, maybe you can join us[br]in the IRC later. If there are any other
0:45:03.469,0:45:08.859
questions, maybe they will look there for[br]a short while or I will forward those
0:45:08.859,0:45:12.239
questions. Right then.[br]J: Thank you.
0:45:12.239,0:45:16.569
H: I thank you very much for the super[br]interesting talk. I learned something new
0:45:16.569,0:45:20.809
about the VPN client that I came into[br]contact with during my work time as well.
0:45:20.809,0:45:28.169
So interesting for me too. And yeah, let's[br]keep it at that.
0:45:28.169,0:45:35.749
postroll music
0:45:35.749,0:45:52.559
Subtitles created by c3subtitles.de[br]in the year 2022. Join, and help us!
0:45:52.559,0:45:59.000
[Translated by {Iikka}{Yli-Kuivila}[br](ITKST56 course assignment at JYU.FI)]