WEBVTT

00:00:02.000 --> 00:00:03.319
hello

00:00:03.319 --> 00:00:06.839
everyone we are getting started here on

00:00:06.839 --> 00:00:09.880
our August lunch and learn session

00:00:09.880 --> 00:00:12.639
presented by Ken group's Atlas customer

00:00:12.639 --> 00:00:16.400
experience team my name is Alice Deane I

00:00:16.400 --> 00:00:19.160
am the engineering manager for the atlas

00:00:19.160 --> 00:00:21.960
customer experience team and I'm excited

00:00:21.960 --> 00:00:24.800
to be presenting this month's session on

00:00:24.800 --> 00:00:28.000
intermediate level Splunk searching so

00:00:28.000 --> 00:00:30.199
thank you all for attending I hope you

00:00:30.199 --> 00:00:33.000
get some good ideas out of this uh and

00:00:33.000 --> 00:00:35.120
certainly encourage engagement through

00:00:35.120 --> 00:00:37.040
the chat uh and I'll have some

00:00:37.040 --> 00:00:39.800
information at the end on following up

00:00:39.800 --> 00:00:42.239
uh and speaking with my team directly on

00:00:42.239 --> 00:00:45.879
any issues uh or interests that you have

00:00:45.879 --> 00:00:48.000
uh around these types of Concepts that

00:00:48.000 --> 00:00:51.520
we're going to cover today so uh jumping

00:00:51.520 --> 00:00:55.199
into an intermediate level uh session I

00:00:55.199 --> 00:00:57.960
I do want to say that we have previously

00:00:57.960 --> 00:01:02.120
done a basic level uh searching uh

00:01:02.120 --> 00:01:05.280
session so that we are really

00:01:05.280 --> 00:01:07.360
progressing from that picking up right

00:01:07.360 --> 00:01:09.400
where we left off uh we've done that

00:01:09.400 --> 00:01:10.640
session with quite a few of our

00:01:10.640 --> 00:01:12.920
customers individually uh and highly

00:01:12.920 --> 00:01:14.640
recommend if you're interested in doing

00:01:14.640 --> 00:01:18.200
that or this session with a larger team

00:01:18.200 --> 00:01:19.920
uh we're happy to to discuss and

00:01:19.920 --> 00:01:22.840
coordinate that uh so getting started

00:01:22.840 --> 00:01:25.600
we're going to take a look at the final

00:01:25.600 --> 00:01:29.000
search uh from our basic search session

00:01:29.000 --> 00:01:31.159
uh and we're going to walk through that

00:01:31.159 --> 00:01:34.159
understand some of the concepts uh and

00:01:34.159 --> 00:01:36.479
then we're going to take a step back

00:01:36.479 --> 00:01:39.479
look a little more generally at SPL

00:01:39.479 --> 00:01:41.759
operations and understanding how

00:01:41.759 --> 00:01:46.200
different commands apply to data uh and

00:01:46.200 --> 00:01:49.320
really that next level of understanding

00:01:49.320 --> 00:01:51.759
for how you can write more complex

00:01:51.759 --> 00:01:54.119
searches and understand really uh when

00:01:54.119 --> 00:01:57.119
to use certain types of commands uh and

00:01:57.119 --> 00:01:59.560
of course uh in the session we're going

00:01:59.560 --> 00:02:04.399
to to have a uh series of demos uh using

00:02:04.399 --> 00:02:07.360
a few specific commands highlighting the

00:02:07.360 --> 00:02:10.440
different SPL command types uh that we

00:02:10.440 --> 00:02:12.840
discuss in the second portion uh and get

00:02:12.840 --> 00:02:15.879
to see that on the tutorial data uh that

00:02:15.879 --> 00:02:18.160
you can also use uh in your environment

00:02:18.160 --> 00:02:20.840
in a test environment uh very

00:02:20.840 --> 00:02:24.200
simply so I will always encourage uh

00:02:24.200 --> 00:02:27.720
especially with search content that you

00:02:27.720 --> 00:02:30.319
look into the additional resource that I

00:02:30.319 --> 00:02:34.120
have listed here the uh search reference

00:02:34.120 --> 00:02:36.440
documentation is one of my favorite

00:02:36.440 --> 00:02:38.760
bookmarks that I use frequently in my

00:02:38.760 --> 00:02:41.000
own environments and working in customer

00:02:41.000 --> 00:02:43.560
environments uh it is really the the

00:02:43.560 --> 00:02:46.000
best quick resource to get information

00:02:46.000 --> 00:02:49.560
on syntax and examples of any Search

00:02:49.560 --> 00:02:51.760
Command uh and is always a great

00:02:51.760 --> 00:02:55.000
resource to have the search manual is a

00:02:55.000 --> 00:02:57.080
little bit more conceptual but as you're

00:02:57.080 --> 00:02:59.120
learning more about different types of

00:02:59.120 --> 00:03:00.360
search operations

00:03:00.360 --> 00:03:02.440
it's very helpful to be able to review

00:03:02.440 --> 00:03:03.360
this

00:03:03.360 --> 00:03:05.560
documentation and have reference

00:03:05.560 --> 00:03:08.680
material that you can come back to uh as

00:03:08.680 --> 00:03:11.080
you are studying and trying to get

00:03:11.080 --> 00:03:13.480
better and writing more complex uh

00:03:13.480 --> 00:03:16.879
search content I have also linked here

00:03:16.879 --> 00:03:18.959
the documentation on how to use the

00:03:18.959 --> 00:03:21.799
Splunk tutorial data uh so if you've not

00:03:21.799 --> 00:03:23.360
done that before it's a very simple

00:03:23.360 --> 00:03:25.920
process uh and there are consistently

00:03:25.920 --> 00:03:28.280
updated download files that spun

00:03:28.280 --> 00:03:30.680
provides uh that you're able to directly

00:03:30.680 --> 00:03:33.439
upload into any spunk environment so

00:03:33.439 --> 00:03:35.560
that's what I'm going to be using today

00:03:35.560 --> 00:03:39.000
uh and given that you are searching over

00:03:39.000 --> 00:03:41.400
uh appropriate time windows for when you

00:03:41.400 --> 00:03:43.920
download the tutorial data set uh these

00:03:43.920 --> 00:03:46.519
searches will work uh on the tutorial

00:03:46.519 --> 00:03:48.760
data as well so highly encourage after

00:03:48.760 --> 00:03:50.879
the fact if you want to go through uh

00:03:50.879 --> 00:03:53.760
and test out some of the content um

00:03:53.760 --> 00:03:56.920
you'll be able to access a recording as

00:03:56.920 --> 00:03:59.360
well as if you'd like the slides that

00:03:59.360 --> 00:04:00.959
I'm Pres presenting off of today which I

00:04:00.959 --> 00:04:02.280
highly encourage because there are a lot

00:04:02.280 --> 00:04:04.799
of useful links in here uh reach out to

00:04:04.799 --> 00:04:06.760
my team again right at the end of the

00:04:06.760 --> 00:04:08.599
slides we'll have that

00:04:08.599 --> 00:04:13.079
info so looking at our overview of basic

00:04:13.079 --> 00:04:15.799
search uh I just want to cover

00:04:15.799 --> 00:04:18.120
conceptually uh the two categories that

00:04:18.120 --> 00:04:21.639
we discuss in that session and so those

00:04:21.639 --> 00:04:24.199
two are the statistical and charting

00:04:24.199 --> 00:04:28.479
functions uh which consist of in those

00:04:28.479 --> 00:04:31.479
demos aggregate and time functions so

00:04:31.479 --> 00:04:33.919
aggregate functions are going to be your

00:04:33.919 --> 00:04:37.400
commonly used statistical functions uh

00:04:37.400 --> 00:04:40.400
meant for summarization uh and then time

00:04:40.400 --> 00:04:43.199
functions actually using uh the

00:04:43.199 --> 00:04:46.639
timestamp field underscore time or any

00:04:46.639 --> 00:04:48.600
other time that you've extracted from

00:04:48.600 --> 00:04:51.759
data uh and looking at earliest latest

00:04:51.759 --> 00:04:55.000
uh relative time values uh in a

00:04:55.000 --> 00:04:58.240
summative fashion and then evaluation

00:04:58.240 --> 00:05:02.320
functions are the separate uh type where

00:05:02.320 --> 00:05:04.400
we discuss comparison and conditional

00:05:04.400 --> 00:05:07.600
statements so using your if and your

00:05:07.600 --> 00:05:10.600
case commands uh in

00:05:10.600 --> 00:05:14.120
evals uh also datetime functions that

00:05:14.120 --> 00:05:17.160
apply operations to events uniquely uh

00:05:17.160 --> 00:05:19.759
so not necessarily summarization but

00:05:19.759 --> 00:05:22.280
interacting with the time values

00:05:22.280 --> 00:05:24.319
themselves maybe changing the time

00:05:24.319 --> 00:05:27.000
format uh and then multivalue evalve

00:05:27.000 --> 00:05:29.360
functions we touch on that very lightly

00:05:29.360 --> 00:05:31.720
uh and it is more conceptual in basic

00:05:31.720 --> 00:05:34.000
search So today we're going to dive in

00:05:34.000 --> 00:05:36.120
as part of our demo and look at

00:05:36.120 --> 00:05:39.160
multivalue eval functions uh later in

00:05:39.160 --> 00:05:41.319
the

00:05:41.479 --> 00:05:44.880
presentation so on this slide here I

00:05:44.880 --> 00:05:48.800
have highlighted uh in Gray the search

00:05:48.800 --> 00:05:52.120
that we end basic search with uh and so

00:05:52.120 --> 00:05:55.000
that is broken up into three segments

00:05:55.000 --> 00:05:57.479
where we have uh the first line being a

00:05:57.479 --> 00:06:00.240
filter to a data set uh this is very

00:06:00.240 --> 00:06:03.120
simply how you are sourcing most of your

00:06:03.120 --> 00:06:06.319
data in most of your searches in Splunk

00:06:06.319 --> 00:06:08.000
uh and we always want to be a specific

00:06:08.000 --> 00:06:11.000
as possible you'll most often see the

00:06:11.000 --> 00:06:13.039
The Logical way to do that is by

00:06:13.039 --> 00:06:15.680
identifying an index and a source type

00:06:15.680 --> 00:06:18.120
possibly some specific values of given

00:06:18.120 --> 00:06:20.199
fields in that data before you start

00:06:20.199 --> 00:06:22.720
applying other operations in our case we

00:06:22.720 --> 00:06:25.199
want to work with a whole data set uh

00:06:25.199 --> 00:06:28.880
and then we mve into applying our eval

00:06:28.880 --> 00:06:30.120
statements

00:06:30.120 --> 00:06:33.080
so in the evals the purpose of these is

00:06:33.080 --> 00:06:36.560
to create some new fields to work with

00:06:36.560 --> 00:06:40.080
uh and so we have two operations here uh

00:06:40.080 --> 00:06:42.440
and you can see that on the first line

00:06:42.440 --> 00:06:46.120
we're starting with an error check field

00:06:46.120 --> 00:06:49.160
uh these are web access logs so we're

00:06:49.160 --> 00:06:52.720
looking at the HTTP status codes as the

00:06:52.720 --> 00:06:56.039
status field and we have a logical

00:06:56.039 --> 00:06:57.599
condition here we greater than or equal

00:06:57.599 --> 00:07:00.680
to 400 we want to return errors and so

00:07:00.680 --> 00:07:04.120
very simple example uh making it as easy

00:07:04.120 --> 00:07:05.879
as possible if you want to get specifics

00:07:05.879 --> 00:07:08.720
on your 200s and your 300s it's the

00:07:08.720 --> 00:07:11.639
exact same type of logic to go and apply

00:07:11.639 --> 00:07:14.120
uh likely a case statement to get some

00:07:14.120 --> 00:07:17.199
additional conditions uh and more unique

00:07:17.199 --> 00:07:20.520
output in an error check or some sort of

00:07:20.520 --> 00:07:23.800
field uh indicating uh what you want to

00:07:23.800 --> 00:07:25.919
see out of your status code so this case

00:07:25.919 --> 00:07:30.080
simple errors or the value of non here

00:07:30.080 --> 00:07:32.120
if we have say at

00:07:32.120 --> 00:07:35.400
200 we're also using a Time function to

00:07:35.400 --> 00:07:39.160
create a second field called day uh you

00:07:39.160 --> 00:07:41.759
may be familiar with some of the uh

00:07:41.759 --> 00:07:46.360
fields that you get out of uh by default

00:07:46.360 --> 00:07:49.759
for most any events in Splunk uh and

00:07:49.759 --> 00:07:51.759
that they're related to breakdowns of

00:07:51.759 --> 00:07:56.000
the time stamps uh you have day month uh

00:07:56.000 --> 00:07:58.240
and many others in this case I want to

00:07:58.240 --> 00:08:00.560
get a specific format for day so we use

00:08:00.560 --> 00:08:03.479
a strif Time function uh and we have a

00:08:03.479 --> 00:08:07.039
time format variable here on the actual

00:08:07.039 --> 00:08:10.280
extracted time stamp first of BL so

00:08:10.280 --> 00:08:12.039
coming out of the second line we've

00:08:12.039 --> 00:08:14.319
accessed our data we have created two

00:08:14.319 --> 00:08:17.479
new fields to use and then we are

00:08:17.479 --> 00:08:20.960
actually performing uh charting with a

00:08:20.960 --> 00:08:23.680
statistical function and so that is

00:08:23.680 --> 00:08:26.240
using time chart and we can see here

00:08:26.240 --> 00:08:29.159
that we are counting our events that

00:08:29.159 --> 00:08:33.479
actually have the error value uh for our

00:08:33.479 --> 00:08:36.000
created error check field and so I'm

00:08:36.000 --> 00:08:39.279
going to Pivot over to uh Splunk here

00:08:39.279 --> 00:08:40.880
and we're going to look at this search

00:08:40.880 --> 00:08:43.440
and I have commented out uh most of the

00:08:43.440 --> 00:08:46.279
logic we'll step back through it uh we

00:08:46.279 --> 00:08:49.200
are looking at our web access log events

00:08:49.200 --> 00:08:52.800
here uh and we want to then apply our

00:08:52.800 --> 00:08:58.240
eval and so by applying the eval we can

00:08:58.240 --> 00:09:01.279
get our error check field that provides

00:09:01.279 --> 00:09:03.279
error or non-error we're seeing that we

00:09:03.279 --> 00:09:05.160
have mostly non-error

00:09:05.160 --> 00:09:09.680
events uh and then we have the day field

00:09:09.680 --> 00:09:11.760
and so day is actually providing the

00:09:11.760 --> 00:09:14.440
full name of day for the time stamp for

00:09:14.440 --> 00:09:17.800
all these events so with our time chart

00:09:17.800 --> 00:09:22.200
this is the summarization uh with a

00:09:22.200 --> 00:09:24.160
condition actually that we're spanning

00:09:24.160 --> 00:09:27.720
by default over a single day so this may

00:09:27.720 --> 00:09:31.839
not be a very logical use of a split by

00:09:31.839 --> 00:09:34.360
day when we are already using a time

00:09:34.360 --> 00:09:37.079
chart command that is dividing our

00:09:37.079 --> 00:09:41.040
results by the time bin uh effectively a

00:09:41.040 --> 00:09:46.079
span of one day but what we can do uh is

00:09:46.079 --> 00:09:50.440
change our split by field to host and

00:09:50.440 --> 00:09:52.600
get a little bit more of a reasonable

00:09:52.600 --> 00:09:54.720
presentation we were able to see with

00:09:54.720 --> 00:09:57.720
the counts in the individual days not

00:09:57.720 --> 00:09:59.600
only split through the time chart but by

00:09:59.600 --> 00:10:02.399
the day field that we only had values

00:10:02.399 --> 00:10:04.959
where our Matrix matched up for the

00:10:04.959 --> 00:10:09.680
actual day so here we have our uh hosts

00:10:09.680 --> 00:10:12.640
one two and three and then across days

00:10:12.640 --> 00:10:15.640
counts of the error events that we

00:10:15.640 --> 00:10:20.160
observe so that is uh the search that we

00:10:20.160 --> 00:10:22.440
end on in basic search the concepts

00:10:22.440 --> 00:10:25.040
there being accessing our data uh

00:10:25.040 --> 00:10:27.279
searching in a descriptive manner using

00:10:27.279 --> 00:10:29.320
our metadata Fields the index and the

00:10:29.320 --> 00:10:32.200
Source type uh the evaluation functions

00:10:32.200 --> 00:10:33.920
where we're creating new Fields

00:10:33.920 --> 00:10:37.639
manipulating data uh and then we have a

00:10:37.639 --> 00:10:40.200
time chart function uh that is providing

00:10:40.200 --> 00:10:42.880
some uh summarized statistics here based

00:10:42.880 --> 00:10:44.480
on the time

00:10:44.480 --> 00:10:48.680
range so we will pivot back and we're

00:10:48.680 --> 00:10:51.399
going to take a step back out of the SPL

00:10:51.399 --> 00:10:54.200
for a second just to talk about these

00:10:54.200 --> 00:10:56.519
different kinds of search operations

00:10:56.519 --> 00:10:59.360
that we just performed so you'll hear

00:10:59.360 --> 00:11:03.079
these terms uh if you are really kind of

00:11:03.079 --> 00:11:06.040
diving deeper into actual operations of

00:11:06.040 --> 00:11:09.920
Splunk searching uh and you can get very

00:11:09.920 --> 00:11:12.560
detailed regarding the optimization of

00:11:12.560 --> 00:11:16.279
searches uh around uh these types of

00:11:16.279 --> 00:11:17.680
commands and the order in which you

00:11:17.680 --> 00:11:21.399
choose to execute SPL today I'm going to

00:11:21.399 --> 00:11:24.240
focus on how these operations actually

00:11:24.240 --> 00:11:27.240
apply to the data and helping you to

00:11:27.240 --> 00:11:29.320
make better decisions about what

00:11:29.320 --> 00:11:32.320
commands are best for the scenario that

00:11:32.320 --> 00:11:34.240
you have or the output that you want to

00:11:34.240 --> 00:11:37.639
see uh and in future sessions we will

00:11:37.639 --> 00:11:39.360
discuss the actual optimization of

00:11:39.360 --> 00:11:42.079
searches uh through this optimal order

00:11:42.079 --> 00:11:46.440
of functions uh and some other means uh

00:11:46.440 --> 00:11:48.200
but just a caveat there that we're going

00:11:48.200 --> 00:11:50.440
to talk uh pretty specifically today

00:11:50.440 --> 00:11:52.839
just about these uh individually how

00:11:52.839 --> 00:11:54.720
they work with data uh and then how you

00:11:54.720 --> 00:11:55.839
see them in

00:11:55.839 --> 00:11:59.839
combination so our types of SPL command

00:11:59.839 --> 00:12:03.160
the top three in bold we'll focus on in

00:12:03.160 --> 00:12:06.079
our examples the first of which is

00:12:06.079 --> 00:12:07.279
streaming

00:12:07.279 --> 00:12:10.760
operations uh which are executed on

00:12:10.760 --> 00:12:13.079
individual events as they return by a

00:12:13.079 --> 00:12:15.399
search uh so you can think of this like

00:12:15.399 --> 00:12:16.120
your

00:12:16.120 --> 00:12:18.880
eval um that is going to be doing

00:12:18.880 --> 00:12:21.440
something to every single event uh

00:12:21.440 --> 00:12:24.279
modifying Fields when they're available

00:12:24.279 --> 00:12:28.399
uh we do have generating functions uh so

00:12:28.399 --> 00:12:30.800
generating function are going to be used

00:12:30.800 --> 00:12:33.839
situationally where you're sourcing data

00:12:33.839 --> 00:12:38.079
from uh non-indexed data sets and so you

00:12:38.079 --> 00:12:40.839
would see that from uh either input

00:12:40.839 --> 00:12:43.760
lookup commands uh or maybe tstats

00:12:43.760 --> 00:12:46.120
pulling information from the tsid X

00:12:46.120 --> 00:12:48.920
Files uh and so generating the

00:12:48.920 --> 00:12:51.079
statistical output based on the data

00:12:51.079 --> 00:12:55.040
available there transforming commands uh

00:12:55.040 --> 00:12:58.560
you will see uh as often as streaming

00:12:58.560 --> 00:13:00.600
commands generally speaking and more

00:13:00.600 --> 00:13:02.800
often than generating commands where

00:13:02.800 --> 00:13:05.399
transforming is intended to order

00:13:05.399 --> 00:13:08.519
results into a data table and I often

00:13:08.519 --> 00:13:11.320
think of this much like how we discuss

00:13:11.320 --> 00:13:13.639
the statistical functions in basic

00:13:13.639 --> 00:13:17.160
search as summarization functions where

00:13:17.160 --> 00:13:19.519
you're looking to condense your overall

00:13:19.519 --> 00:13:22.680
data set uh into really manageable

00:13:22.680 --> 00:13:24.880
consumable results uh so these

00:13:24.880 --> 00:13:28.320
operations that apply that summarization

00:13:28.320 --> 00:13:31.720
are transform perform we do have two

00:13:31.720 --> 00:13:35.600
additional types of SPL commands uh the

00:13:35.600 --> 00:13:39.480
first is orchestrating uh you can read

00:13:39.480 --> 00:13:41.680
about these I will not discuss in great

00:13:41.680 --> 00:13:45.199
detail uh they are used to manipulate

00:13:45.199 --> 00:13:48.639
how searches are actually U processed or

00:13:48.639 --> 00:13:50.800
or how commands are processed uh and

00:13:50.800 --> 00:13:54.079
they don't directly affect the results

00:13:54.079 --> 00:13:56.079
in a search how we think about say

00:13:56.079 --> 00:13:59.839
applying a stats or an eval uh to a data

00:13:59.839 --> 00:14:02.320
set uh so if you're interested

00:14:02.320 --> 00:14:04.399
definitely check it out uh link

00:14:04.399 --> 00:14:07.720
documentation has details there um data

00:14:07.720 --> 00:14:11.120
set processing is seen much more often

00:14:11.120 --> 00:14:15.000
uh and you do have uh some conditional

00:14:15.000 --> 00:14:18.680
uh scenarios where commands can act as

00:14:18.680 --> 00:14:21.759
data set processing so the uh

00:14:21.759 --> 00:14:23.959
distinction for data set processing is

00:14:23.959 --> 00:14:26.360
going to be that you are operating in

00:14:26.360 --> 00:14:29.800
bulk on a single completed data set at

00:14:29.800 --> 00:14:32.240
one time so we'll we'll look at an

00:14:32.240 --> 00:14:33.920
example of

00:14:33.920 --> 00:14:36.600
that I want to Pivot back to our main

00:14:36.600 --> 00:14:38.360
three that we're going to be focusing on

00:14:38.360 --> 00:14:39.839
and I have mentioned some of these

00:14:39.839 --> 00:14:43.800
examples already uh the eval functions

00:14:43.800 --> 00:14:45.880
that we've been talking about so far are

00:14:45.880 --> 00:14:47.920
perfect examples of our streaming

00:14:47.920 --> 00:14:51.440
commands uh so where we are creating new

00:14:51.440 --> 00:14:55.600
fields for each entry or log event uh

00:14:55.600 --> 00:14:59.399
where we are modifying values for all of

00:14:59.399 --> 00:15:01.920
the results that are available uh that

00:15:01.920 --> 00:15:05.279
is where we are streaming um with the

00:15:05.279 --> 00:15:08.560
search functions input lookup is

00:15:08.560 --> 00:15:09.959
possibly one of the most common

00:15:09.959 --> 00:15:12.399
generating commands that I see uh

00:15:12.399 --> 00:15:15.199
because someone is intending to uh

00:15:15.199 --> 00:15:18.720
Source a data set stored in a CSV file

00:15:18.720 --> 00:15:21.480
or a KV store collection uh and you're

00:15:21.480 --> 00:15:23.720
able to bring that back as a report and

00:15:23.720 --> 00:15:26.959
use that logic uh in your

00:15:26.959 --> 00:15:29.639
queries so that is

00:15:29.639 --> 00:15:33.399
uh not requiring the index data uh or

00:15:33.399 --> 00:15:35.560
any index data to actually return the

00:15:35.560 --> 00:15:38.120
results that you want to

00:15:38.120 --> 00:15:41.319
see and we've talked about stats very

00:15:41.319 --> 00:15:43.600
generally speaking uh with a lot of

00:15:43.600 --> 00:15:46.440
unique functions you can apply there uh

00:15:46.440 --> 00:15:49.560
where this is going to provide a tabular

00:15:49.560 --> 00:15:53.560
output uh and is serving that purpose of

00:15:53.560 --> 00:15:54.800
summarization so we're really

00:15:54.800 --> 00:15:57.560
reformatting the data uh into that

00:15:57.560 --> 00:16:00.920
tabular report

00:16:02.000 --> 00:16:06.519
so we see in this example search here uh

00:16:06.519 --> 00:16:09.000
that we are often combining these

00:16:09.000 --> 00:16:12.360
different types of search operations so

00:16:12.360 --> 00:16:15.240
in this example that we have uh I have

00:16:15.240 --> 00:16:19.319
data that already exists in a CSV file

00:16:19.319 --> 00:16:22.839
we are applying a streaming command here

00:16:22.839 --> 00:16:26.000
uh where evaluating each line to see if

00:16:26.000 --> 00:16:28.399
we match a condition and then returning

00:16:28.399 --> 00:16:29.639
the results

00:16:29.639 --> 00:16:32.240
based on that evaluation and then we're

00:16:32.240 --> 00:16:34.199
applying a transforming command at the

00:16:34.199 --> 00:16:36.639
end which is that stats summarization

00:16:36.639 --> 00:16:40.480
getting the maximum values uh for the uh

00:16:40.480 --> 00:16:44.319
count of errors and the host that is

00:16:44.319 --> 00:16:47.600
associated with that so let's PIV over

00:16:47.600 --> 00:16:52.079
to Splunk and we'll take a look at that

00:16:54.160 --> 00:16:56.319
example so I'm just going to grab my

00:16:56.319 --> 00:16:59.440
search here and I pre- commented out

00:16:59.440 --> 00:17:03.519
uh the specific uh lines following input

00:17:03.519 --> 00:17:06.079
lookup just to see that this generating

00:17:06.079 --> 00:17:07.799
command here is not looking for any

00:17:07.799 --> 00:17:10.160
specific index data uh we're pulling

00:17:10.160 --> 00:17:13.240
directly the results that I have in a

00:17:13.240 --> 00:17:17.720
CSV file uh here into this output and so

00:17:17.720 --> 00:17:20.520
we have a count of Errors observed

00:17:20.520 --> 00:17:25.439
across multiple hosts our where command

00:17:25.439 --> 00:17:28.520
uh you might think is reformatting data

00:17:28.520 --> 00:17:31.000
in this sense it it is transforming the

00:17:31.000 --> 00:17:34.160
results but the evaluation of a wear

00:17:34.160 --> 00:17:37.320
function does apply effectively to every

00:17:37.320 --> 00:17:41.760
event that is returned uh so it is u a

00:17:41.760 --> 00:17:43.960
streaming command that is going to

00:17:43.960 --> 00:17:46.559
filter down our result set based on our

00:17:46.559 --> 00:17:49.120
condition that the error count is less

00:17:49.120 --> 00:17:50.919
than

00:17:50.919 --> 00:17:54.760
200 so the following line is our

00:17:54.760 --> 00:17:57.320
transforming command where we have two

00:17:57.320 --> 00:18:02.240
results left uh 187 for host 3 we want

00:18:02.240 --> 00:18:06.039
to see our maximum values here of 187 on

00:18:06.039 --> 00:18:09.960
host 3 so our scenario here has really

00:18:09.960 --> 00:18:13.400
uh covered where you may have uh hosts

00:18:13.400 --> 00:18:15.960
that are trending toward a negative

00:18:15.960 --> 00:18:19.280
State you're aware that uh the second

00:18:19.280 --> 00:18:22.039
host had already exceeded its uh

00:18:22.039 --> 00:18:25.360
threshold value for errors but host 3

00:18:25.360 --> 00:18:27.440
also appears to be trending toward this

00:18:27.440 --> 00:18:30.159
threshold uh so being able to combine

00:18:30.159 --> 00:18:33.000
these types of commands uh understand

00:18:33.000 --> 00:18:35.240
the logical condition that you're

00:18:35.240 --> 00:18:37.679
searching for uh and then also providing

00:18:37.679 --> 00:18:40.840
that consumable output uh so combining

00:18:40.840 --> 00:18:44.480
all three of our types of commands

00:18:45.320 --> 00:18:49.440
here so uh I'm going to jump to an SPL

00:18:49.440 --> 00:18:53.159
demo and as I go through these different

00:18:53.159 --> 00:18:55.840
commands uh I'm going to be referencing

00:18:55.840 --> 00:18:58.360
back to the different command types that

00:18:58.360 --> 00:19:00.080
we're working with I'm going to

00:19:00.080 --> 00:19:02.360
introduce in a lot of these searches uh

00:19:02.360 --> 00:19:04.679
a lot of small commands uh that I won't

00:19:04.679 --> 00:19:07.000
talk about in great detail and that

00:19:07.000 --> 00:19:09.360
really is the purpose of using your

00:19:09.360 --> 00:19:11.640
search manual uh using your search

00:19:11.640 --> 00:19:14.760
reference documentation uh so I will

00:19:14.760 --> 00:19:17.400
glance over the use case uh talk about

00:19:17.400 --> 00:19:19.559
how it's meant to be applied and then

00:19:19.559 --> 00:19:22.200
using in your own scenarios uh where you

00:19:22.200 --> 00:19:24.400
have problem you need to solve uh

00:19:24.400 --> 00:19:26.880
referencing the docs to find out where

00:19:26.880 --> 00:19:29.960
you can apply uh similar functions to

00:19:29.960 --> 00:19:32.559
what we observe in the the demonstration

00:19:32.559 --> 00:19:36.760
here so the First Command I'm going to

00:19:36.760 --> 00:19:40.880
focus on is the Rex command so Rex is a

00:19:40.880 --> 00:19:43.480
streaming command that you often see

00:19:43.480 --> 00:19:46.559
applied to data sets that do not fully

00:19:46.559 --> 00:19:49.720
have data extracted in the format that

00:19:49.720 --> 00:19:53.159
you want to be using um in your

00:19:53.159 --> 00:19:56.760
reporting or in your logic uh and so

00:19:56.760 --> 00:20:00.120
this could very well be handled actually

00:20:00.120 --> 00:20:03.440
in the uh configuration of props and

00:20:03.440 --> 00:20:06.080
transforms and extracting fields at the

00:20:06.080 --> 00:20:08.480
right times and indexing data but as

00:20:08.480 --> 00:20:10.280
your bringing new data sources you need

00:20:10.280 --> 00:20:12.480
to understand what's available for use

00:20:12.480 --> 00:20:14.360
in spunk a lot of times you'll find

00:20:14.360 --> 00:20:16.840
yourself needing to extract new fields

00:20:16.840 --> 00:20:19.200
in line in your searches uh and be able

00:20:19.200 --> 00:20:22.080
to use those in your search Logic Rex

00:20:22.080 --> 00:20:28.039
also has uh a said mode that I also see

00:20:28.039 --> 00:20:31.600
testing done for masking of data in line

00:20:31.600 --> 00:20:34.080
prior to actually putting that into

00:20:34.080 --> 00:20:35.120
indexing

00:20:35.120 --> 00:20:38.000
configurations um so Rex you would

00:20:38.000 --> 00:20:41.200
generally see used um when you don't

00:20:41.200 --> 00:20:43.039
have those fields available you need to

00:20:43.039 --> 00:20:45.640
use them at that time uh and then we're

00:20:45.640 --> 00:20:47.120
going to take a look at an example of

00:20:47.120 --> 00:20:49.640
masking data as well uh to test your

00:20:49.640 --> 00:20:53.480
Syntax for a said style replace uh in

00:20:53.480 --> 00:21:00.600
config files so we will jump back over

00:21:04.679 --> 00:21:06.880
so I'm going to start with a search on

00:21:06.880 --> 00:21:10.120
an index Source type uh my tutorial data

00:21:10.120 --> 00:21:13.159
and then this is actual uh Linux secure

00:21:13.159 --> 00:21:16.159
logging uh so these are going to be OS

00:21:16.159 --> 00:21:19.039
security logs and we're looking at all

00:21:19.039 --> 00:21:21.039
of our web hosts uh that we've been

00:21:21.039 --> 00:21:22.440
focusing on

00:21:22.440 --> 00:21:25.000
previously in our events you can see

00:21:25.000 --> 00:21:29.039
that we have uh first here uh an EV that

00:21:29.039 --> 00:21:31.720
has fail password for invalid user in

00:21:31.720 --> 00:21:34.320
that we're provided a source IP a source

00:21:34.320 --> 00:21:36.559
port and we go to see the fields that

00:21:36.559 --> 00:21:38.919
are extracted and that's that's not

00:21:38.919 --> 00:21:41.919
being done for us automatically so just

00:21:41.919 --> 00:21:43.880
to start testing our logic to see if we

00:21:43.880 --> 00:21:46.799
can get uh the results we want to see

00:21:46.799 --> 00:21:49.760
we're going to use the Rex command and

00:21:49.760 --> 00:21:53.240
in doing so we are applying this

00:21:53.240 --> 00:21:55.440
operation across every event again a

00:21:55.440 --> 00:21:59.600
streaming command we are looking at the

00:21:59.600 --> 00:22:01.279
raw field so we're actually looking at

00:22:01.279 --> 00:22:04.679
the raw text of each of these log events

00:22:04.679 --> 00:22:07.480
and then the rec syntax is simply to

00:22:07.480 --> 00:22:11.960
provide in double quotes uh a Rex uh

00:22:11.960 --> 00:22:14.840
match and we're using named groups for

00:22:14.840 --> 00:22:17.440
field extractions so for every single

00:22:17.440 --> 00:22:19.440
event that we see failed password for

00:22:19.440 --> 00:22:22.919
invalid user we are actually extracting

00:22:22.919 --> 00:22:26.400
a user field The Source IP field and the

00:22:26.400 --> 00:22:28.799
source Port field for the sake of

00:22:28.799 --> 00:22:30.880
Simplicity I tried to keep the RX simple

00:22:30.880 --> 00:22:33.760
you can make this as complex as you need

00:22:33.760 --> 00:22:37.679
to for your needs for your data uh and

00:22:37.679 --> 00:22:40.960
so in our extracted Fields uh I've

00:22:40.960 --> 00:22:42.840
actually pre-selected these so we can

00:22:42.840 --> 00:22:46.240
see our user is now available and this

00:22:46.240 --> 00:22:50.039
applies to the events where the Rex was

00:22:50.039 --> 00:22:53.159
actually valid and matching on the uh

00:22:53.159 --> 00:22:57.440
failed password for invalid user Etc

00:22:57.440 --> 00:23:00.120
string so now that we have our Fields

00:23:00.120 --> 00:23:03.799
extracted we can actually use these and

00:23:03.799 --> 00:23:04.799
we want

00:23:04.799 --> 00:23:09.400
to do a stats count as failed login so

00:23:09.400 --> 00:23:13.400
anytime you see uh a an operation as and

00:23:13.400 --> 00:23:16.640
then a unique name just a rename uh

00:23:16.640 --> 00:23:19.080
through the transformation function

00:23:19.080 --> 00:23:21.480
easier way to uh actually keep

00:23:21.480 --> 00:23:23.480
consistency uh with referencing your

00:23:23.480 --> 00:23:26.760
Fields as well as not have to rename

00:23:26.760 --> 00:23:29.919
later on uh with some additional in this

00:23:29.919 --> 00:23:31.679
case you'd have to reference the name

00:23:31.679 --> 00:23:34.520
distinct count uh so just a way to keep

00:23:34.520 --> 00:23:38.320
things clean and easy to use in further

00:23:38.320 --> 00:23:42.159
uh lines of SPL so we are counting our

00:23:42.159 --> 00:23:43.919
failed logins we're looking at the

00:23:43.919 --> 00:23:47.840
distinct count of the source IP values

00:23:47.840 --> 00:23:50.000
that we have and then we're splitting

00:23:50.000 --> 00:23:52.960
that by the host and the user so you can

00:23:52.960 --> 00:23:55.720
see here uh this tutorial data is

00:23:55.720 --> 00:23:57.880
actually pretty flat across most of the

00:23:57.880 --> 00:24:00.120
sources so we're not going to have uh

00:24:00.120 --> 00:24:04.679
any outliers or spikes in our stats here

00:24:04.679 --> 00:24:07.960
but you can see the resulting

00:24:08.960 --> 00:24:11.440
presentation in line four we do have a

00:24:11.440 --> 00:24:14.840
sort command and this is an example of a

00:24:14.840 --> 00:24:17.520
data set processing command where we are

00:24:17.520 --> 00:24:20.400
actually evaluating a full completed

00:24:20.400 --> 00:24:23.640
data set and reordering it uh given the

00:24:23.640 --> 00:24:26.000
logic here we want to descend on these

00:24:26.000 --> 00:24:29.000
numeric values uh so keep mind as you're

00:24:29.000 --> 00:24:31.200
operating on different fields it's going

00:24:31.200 --> 00:24:33.799
to be the same sort of either basic

00:24:33.799 --> 00:24:37.159
numeric or the lexicographical ordering

00:24:37.159 --> 00:24:40.360
that you typically see in

00:24:40.840 --> 00:24:45.720
Splunk so we do have a second example uh

00:24:45.720 --> 00:24:49.200
with the said style

00:24:54.240 --> 00:24:58.640
replace so you can see in my events here

00:24:58.640 --> 00:25:01.640
uh we are searching the tutorial and

00:25:01.640 --> 00:25:05.039
vendor sales index and Source type and

00:25:05.039 --> 00:25:06.720
I've gone ahead and applied one

00:25:06.720 --> 00:25:09.399
operation and this is going to be a

00:25:09.399 --> 00:25:11.880
helpful operation to understand really

00:25:11.880 --> 00:25:14.679
what we are replacing and how to get

00:25:14.679 --> 00:25:18.159
consistent operation on these fields uh

00:25:18.159 --> 00:25:20.279
so in this case we are actually creating

00:25:20.279 --> 00:25:23.559
an ID length field where we are going to

00:25:23.559 --> 00:25:26.960
choose to mask the value of account ID

00:25:26.960 --> 00:25:29.120
in our Rex command we want to know that

00:25:29.120 --> 00:25:31.679
that's a consistent number of characters

00:25:31.679 --> 00:25:33.799
uh through all of our data it's very

00:25:33.799 --> 00:25:37.080
simple to spot check uh but just to be

00:25:37.080 --> 00:25:39.440
certain we want to apply this to all of

00:25:39.440 --> 00:25:42.760
our data in this case streaming command

00:25:42.760 --> 00:25:45.520
uh through this eval uh we

00:25:45.520 --> 00:25:49.279
are uh changing the type of the data

00:25:49.279 --> 00:25:51.919
because account ID is actually numeric

00:25:51.919 --> 00:25:53.720
we're making that a string value so that

00:25:53.720 --> 00:25:56.720
we can look at the length these are

00:25:56.720 --> 00:25:58.840
common functions in any programming

00:25:58.840 --> 00:26:01.559
languages uh and so the syntax here in

00:26:01.559 --> 00:26:04.039
SPL is quite simple uh just to be able

00:26:04.039 --> 00:26:06.520
to get that contextual feeli we

00:26:06.520 --> 00:26:09.399
understand we have 16 characters for

00:26:09.399 --> 00:26:12.480
100% of our events in the account

00:26:12.480 --> 00:26:17.000
IDs so actually applying our Rex command

00:26:17.000 --> 00:26:20.760
we are going to now specify a unique

00:26:20.760 --> 00:26:23.919
field not just uncore raw uh we are

00:26:23.919 --> 00:26:27.159
applying the said mode and this is a

00:26:27.159 --> 00:26:30.799
said syntax uh replacement uh looking

00:26:30.799 --> 00:26:33.559
for the uh it's a capture group for the

00:26:33.559 --> 00:26:35.880
first 12 digits uh and then we're

00:26:35.880 --> 00:26:39.240
replacing that with a series of 12 X's

00:26:39.240 --> 00:26:42.039
so you can see in our first event the

00:26:42.039 --> 00:26:45.320
account ID is now masked we only have uh

00:26:45.320 --> 00:26:48.520
the remaining four digits to be able to

00:26:48.520 --> 00:26:52.320
identify that and so if our data was

00:26:52.320 --> 00:26:55.360
indexed and is appropriately done so uh

00:26:55.360 --> 00:26:58.039
in Splunk with the full account IDs but

00:26:58.039 --> 00:27:00.360
for for the sake of reporting we want to

00:27:00.360 --> 00:27:04.840
be able to mask that um for the audience

00:27:04.840 --> 00:27:07.799
then we're able to use the the said

00:27:07.799 --> 00:27:11.919
replace and then to finalize a report

00:27:11.919 --> 00:27:13.880
this is just an example of the top

00:27:13.880 --> 00:27:16.399
command which does a few operations

00:27:16.399 --> 00:27:18.120
together uh and makes for a good

00:27:18.120 --> 00:27:20.720
shorthand report uh taking all the

00:27:20.720 --> 00:27:24.080
unique values of the provided field uh

00:27:24.080 --> 00:27:26.480
giving you a count of those values and

00:27:26.480 --> 00:27:29.000
then showing the percentage

00:27:29.000 --> 00:27:31.679
of the makeup for the total data set

00:27:31.679 --> 00:27:34.520
that that unique value accounts for so

00:27:34.520 --> 00:27:37.399
again pretty flat in this tutorial data

00:27:37.399 --> 00:27:40.200
in seeing a very consistent

00:27:40.200 --> 00:27:45.159
.3% uh across these different account

00:27:46.679 --> 00:27:51.080
IDs so we have looked at a few examples

00:27:51.080 --> 00:27:54.640
with the Rex command uh and that is

00:27:54.640 --> 00:27:57.039
again streaming we're going to look at

00:27:57.039 --> 00:27:59.120
another streaming command

00:27:59.120 --> 00:28:02.399
uh which is going to be a set of

00:28:02.399 --> 00:28:07.200
multivalue eval functions and so again

00:28:07.200 --> 00:28:09.559
if you're to have a bookmark for search

00:28:09.559 --> 00:28:12.320
documentation multivalue eval functions

00:28:12.320 --> 00:28:14.559
are a great one to have uh because when

00:28:14.559 --> 00:28:17.240
you encounter these uh it really takes

00:28:17.240 --> 00:28:19.960
some time to figure out how to actually

00:28:19.960 --> 00:28:25.960
operate on data um and so the U

00:28:25.960 --> 00:28:29.559
multivalue functions are um really just

00:28:29.559 --> 00:28:31.799
a collection that depending on your use

00:28:31.799 --> 00:28:34.679
case uh you're able to determine the the

00:28:34.679 --> 00:28:39.080
best to apply um you see it often used

00:28:39.080 --> 00:28:42.840
with uh Json and XML so data formats

00:28:42.840 --> 00:28:44.880
that are actually naturally going to

00:28:44.880 --> 00:28:47.360
provide uh a multivalue field where you

00:28:47.360 --> 00:28:50.480
have repeated tags or Keys uh across

00:28:50.480 --> 00:28:54.320
unique uh events as they're extracted uh

00:28:54.320 --> 00:28:56.360
and you often see a lot of times in

00:28:56.360 --> 00:28:58.480
Windows event logs you actually have

00:28:58.480 --> 00:29:01.360
repeated key values uh where your values

00:29:01.360 --> 00:29:02.960
are different and the position in the

00:29:02.960 --> 00:29:05.200
event is actually specific to a

00:29:05.200 --> 00:29:08.840
condition uh so you may have um a need

00:29:08.840 --> 00:29:11.440
for extraction or interaction with one

00:29:11.440 --> 00:29:14.399
of those unique values uh to actually

00:29:14.399 --> 00:29:18.600
get a reasonable outcome from your

00:29:18.600 --> 00:29:22.799
data and so um we're going to use

00:29:22.799 --> 00:29:25.960
multivalue eval functions uh when we

00:29:25.960 --> 00:29:28.679
have a uh change we want to the

00:29:28.679 --> 00:29:31.880
presentation of data uh and we're able

00:29:31.880 --> 00:29:34.880
to do so with multivalue Fields this I

00:29:34.880 --> 00:29:36.720
would say often occurs when you have

00:29:36.720 --> 00:29:39.960
multivalue data uh and then you want to

00:29:39.960 --> 00:29:43.080
be able to change the the format of the

00:29:43.080 --> 00:29:45.640
multivalue fields there uh and then

00:29:45.640 --> 00:29:46.960
we're also going to look at a quick

00:29:46.960 --> 00:29:51.279
example of uh actually using multivalue

00:29:51.279 --> 00:29:54.880
evaluation uh as a logical

00:29:54.880 --> 00:30:00.039
condition so uh the first

00:30:03.320 --> 00:30:05.679
example we're going to start with a

00:30:05.679 --> 00:30:08.720
simple table looking at our web access

00:30:08.720 --> 00:30:11.240
logs uh and so we're just going to pull

00:30:11.240 --> 00:30:14.880
in our status and refer domain fields

00:30:14.880 --> 00:30:18.440
and so you can see uh we've got a uh

00:30:18.440 --> 00:30:23.000
HTTP status code uh and we've got uh the

00:30:23.000 --> 00:30:26.120
format of a protocol subdomain uh domain

00:30:26.120 --> 00:30:29.519
tldd and our scenario here is that for a

00:30:29.519 --> 00:30:31.559
Simplicity of reporting uh we just want

00:30:31.559 --> 00:30:33.760
to work with this referred domain field

00:30:33.760 --> 00:30:38.320
and be able to simplify that so in

00:30:38.320 --> 00:30:41.799
actually splitting out the field in this

00:30:41.799 --> 00:30:44.880
case uh split refer domain and then

00:30:44.880 --> 00:30:47.720
choosing the period character as our

00:30:47.720 --> 00:30:50.399
point to split the data we're creating a

00:30:50.399 --> 00:30:52.919
multivalue uh from what was previously

00:30:52.919 --> 00:30:57.200
just a a single value field uh and using

00:30:57.200 --> 00:31:01.600
this we can actually create a new field

00:31:01.600 --> 00:31:06.080
by using the index of a multivalue field

00:31:06.080 --> 00:31:08.039
and in this case uh we're looking at

00:31:08.039 --> 00:31:09.760
index

00:31:09.760 --> 00:31:13.279
012 the multivalue index function allows

00:31:13.279 --> 00:31:15.799
us to Target a specific field and then

00:31:15.799 --> 00:31:18.559
choose a starting and ending index to

00:31:18.559 --> 00:31:21.320
extract given values there are a number

00:31:21.320 --> 00:31:23.320
of ways to do this in our case here

00:31:23.320 --> 00:31:25.039
where we have three entries it's quite

00:31:25.039 --> 00:31:26.639
simple just to give that start and end

00:31:26.639 --> 00:31:28.639
of range as the

00:31:28.639 --> 00:31:30.039
two entries

00:31:30.039 --> 00:31:35.360
apart so as we are working to recreate

00:31:35.360 --> 00:31:39.200
our domain and so that is just applying

00:31:39.200 --> 00:31:41.720
uh for this new domain field we have

00:31:41.720 --> 00:31:44.200
Buttercup games.com and what was

00:31:44.200 --> 00:31:48.200
previously the HTTP www. Buttercup

00:31:48.200 --> 00:31:51.440
games.com uh we can now use those fields

00:31:51.440 --> 00:31:54.720
in a transformation function in this

00:31:54.720 --> 00:31:58.039
case simple stats count by status uh in

00:31:58.039 --> 00:32:00.200
the

00:32:02.600 --> 00:32:06.960
domain so I do want to look at another

00:32:06.960 --> 00:32:10.240
uh example here that is similar but

00:32:10.240 --> 00:32:13.639
we're going to use a multivalue function

00:32:13.639 --> 00:32:16.919
to actually test a condition and so I'm

00:32:16.919 --> 00:32:18.399
going

00:32:18.399 --> 00:32:21.639
to in this case uh be searching the same

00:32:21.639 --> 00:32:24.240
data we're going to start with a stats

00:32:24.240 --> 00:32:28.639
command and so a stats count as well as

00:32:28.639 --> 00:32:32.039
a values of status and so the values

00:32:32.039 --> 00:32:33.360
function is going to provide all the

00:32:33.360 --> 00:32:37.480
unique values of a given field uh based

00:32:37.480 --> 00:32:41.840
on uh the split by and so that produces

00:32:41.840 --> 00:32:44.960
a multivalue field here in the case of

00:32:44.960 --> 00:32:47.279
status we have quite a few events uh

00:32:47.279 --> 00:32:50.799
that have multiple status codes and as

00:32:50.799 --> 00:32:52.960
we're interested in pulling those events

00:32:52.960 --> 00:32:57.480
out we can use an MV count function to

00:32:57.480 --> 00:33:01.200
eval valate and filter our data set to

00:33:01.200 --> 00:33:04.240
those specific events so a very simple

00:33:04.240 --> 00:33:07.200
operation here just looking at what has

00:33:07.200 --> 00:33:10.240
the uh what has more than a single value

00:33:10.240 --> 00:33:13.399
for status uh but very useful as you're

00:33:13.399 --> 00:33:15.919
applying this in reporting especially in

00:33:15.919 --> 00:33:19.000
combination with others and uh with more

00:33:19.000 --> 00:33:22.639
complex conditions

00:33:22.639 --> 00:33:28.200
um so uh that is our set of multivalue

00:33:28.200 --> 00:33:32.519
eval functions there as streaming

00:33:34.240 --> 00:33:38.279
commands so for a uh final section of

00:33:38.279 --> 00:33:42.000
the demo I want to talk about a concept

00:33:42.000 --> 00:33:44.720
that is not so much a set of functions

00:33:44.720 --> 00:33:47.960
uh but really enables uh more complex

00:33:47.960 --> 00:33:50.159
and interesting searching and can allow

00:33:50.159 --> 00:33:52.799
us to use a few different types of

00:33:52.799 --> 00:33:57.240
commands uh in our SPL and so concept of

00:33:57.240 --> 00:34:00.200
sub searching for both filtering and

00:34:00.200 --> 00:34:04.279
enrichment uh is taking secondary search

00:34:04.279 --> 00:34:06.960
results uh and we're using that to

00:34:06.960 --> 00:34:10.359
affect a primary search uh so a sub

00:34:10.359 --> 00:34:12.200
search will be executed the results

00:34:12.200 --> 00:34:15.079
returned and depending on how it's used

00:34:15.079 --> 00:34:17.760
uh this is going to be processed in the

00:34:17.760 --> 00:34:21.599
original search uh and that is going to

00:34:21.599 --> 00:34:24.359
will look at an example that it is

00:34:24.359 --> 00:34:27.399
filtering So based on the results we get

00:34:27.399 --> 00:34:31.240
a effectively a value equals X or value

00:34:31.240 --> 00:34:34.320
equals y uh for one of our fields that

00:34:34.320 --> 00:34:37.159
we're looking at in the sub search uh

00:34:37.159 --> 00:34:39.320
and then we're also going to look at an

00:34:39.320 --> 00:34:42.399
enrichment example so you see this often

00:34:42.399 --> 00:34:45.760
when you have uh a data set maybe saved

00:34:45.760 --> 00:34:48.480
in a lookup table uh or you just have a

00:34:48.480 --> 00:34:50.079
simple reference where you want to bring

00:34:50.079 --> 00:34:52.879
in more context maybe descriptions of

00:34:52.879 --> 00:34:54.560
event codes things like

00:34:54.560 --> 00:34:59.640
that so in that case

00:35:02.160 --> 00:35:05.440
we'll look at the First Command here now

00:35:05.440 --> 00:35:08.160
I'm going to run my search and we're

00:35:08.160 --> 00:35:12.119
going to Pivot over uh to a sub search

00:35:12.119 --> 00:35:14.480
tab here and so you can see our sub

00:35:14.480 --> 00:35:19.720
search looking at the secure uh logs uh

00:35:19.720 --> 00:35:21.880
we are actually just pulling out the

00:35:21.880 --> 00:35:24.359
search to see what the results are uh or

00:35:24.359 --> 00:35:26.079
what's going to be returned from that

00:35:26.079 --> 00:35:28.839
sub search so we're applying the same

00:35:28.839 --> 00:35:31.200
rex that we had before to extract our

00:35:31.200 --> 00:35:33.720
Fields we're applying a wear a streaming

00:35:33.720 --> 00:35:35.920
command looking for anything that's not

00:35:35.920 --> 00:35:38.599
null for user we observed that we had

00:35:38.599 --> 00:35:40.920
about 60% of our events that were going

00:35:40.920 --> 00:35:43.359
to be null based on not having a user

00:35:43.359 --> 00:35:46.800
field and so looking at that total data

00:35:46.800 --> 00:35:50.280
set uh we're just going to count by our

00:35:50.280 --> 00:35:53.839
source IP and this is often a quick way

00:35:53.839 --> 00:35:56.839
to really just get a list of unique

00:35:56.839 --> 00:35:59.880
values of any given field uh and then

00:35:59.880 --> 00:36:03.119
operating on that uh to return just the

00:36:03.119 --> 00:36:05.079
the list of values few different ways to

00:36:05.079 --> 00:36:08.800
do that uh see stats count pretty often

00:36:08.800 --> 00:36:10.599
and in this case we're actually tbling

00:36:10.599 --> 00:36:13.960
out just keeping our source IP field and

00:36:13.960 --> 00:36:16.800
renaming to client IP so the resulting

00:36:16.800 --> 00:36:20.560
data set is a single column table uh

00:36:20.560 --> 00:36:21.440
with

00:36:21.440 --> 00:36:26.319
182 results and the field name is client

00:36:26.319 --> 00:36:29.880
IP so so when returned to the original

00:36:29.880 --> 00:36:32.119
search we're running this as a sub

00:36:32.119 --> 00:36:36.319
search the effective result of this is

00:36:36.319 --> 00:36:39.960
actually client IP equals my first value

00:36:39.960 --> 00:36:43.800
here or client IP equals my second value

00:36:43.800 --> 00:36:46.960
and so on through the full data set and

00:36:46.960 --> 00:36:49.200
so looking at our search here we're

00:36:49.200 --> 00:36:52.359
applying this to the access logs you can

00:36:52.359 --> 00:36:55.280
see that we had a field named Source IP

00:36:55.280 --> 00:36:58.520
in the secure logs uh and we renamed a

00:36:58.520 --> 00:37:02.160
client IP so that we could apply this to

00:37:02.160 --> 00:37:05.760
the access logs where client IP is the

00:37:05.760 --> 00:37:09.480
actual field name for the uh Source IP

00:37:09.480 --> 00:37:13.560
data and in this case we are filtering

00:37:13.560 --> 00:37:16.079
to the client IPS relevant in the secure

00:37:16.079 --> 00:37:19.839
logs for our web access

00:37:19.839 --> 00:37:23.960
logs so uncommenting here we have a

00:37:23.960 --> 00:37:26.800
series of operations that we're doing uh

00:37:26.800 --> 00:37:29.000
and I'm just going to run the mall at

00:37:29.000 --> 00:37:33.079
once and talk through uh that we are

00:37:33.079 --> 00:37:37.240
counting uh the status or we're counting

00:37:37.240 --> 00:37:40.319
the events by status and client IP uh

00:37:40.319 --> 00:37:42.640
for the client IPS that were relevant to

00:37:42.640 --> 00:37:44.880
authentication failures in the secure

00:37:44.880 --> 00:37:48.760
logs we are then creating a status count

00:37:48.760 --> 00:37:52.040
field just by combining uh our status

00:37:52.040 --> 00:37:54.680
and count Fields uh adding a colant

00:37:54.680 --> 00:37:58.640
between them uh and then we are doing a

00:37:58.640 --> 00:38:02.079
second uh stats statement here to

00:38:02.079 --> 00:38:03.960
actually combine all of our newly

00:38:03.960 --> 00:38:06.319
created Fields together in a more

00:38:06.319 --> 00:38:10.560
condensed report so transforming command

00:38:10.560 --> 00:38:12.520
then streaming for creating our new

00:38:12.520 --> 00:38:15.359
field another transforming command and

00:38:15.359 --> 00:38:17.880
then our sort for data set processing

00:38:17.880 --> 00:38:20.920
actually gives us the results here for a

00:38:20.920 --> 00:38:25.480
given client IP and so we are in this

00:38:25.480 --> 00:38:28.440
case looking for the scenario that

00:38:28.440 --> 00:38:31.319
these client IPS that are involved in

00:38:31.319 --> 00:38:34.240
authentication failures to the web

00:38:34.240 --> 00:38:37.319
servers in this case these were all over

00:38:37.319 --> 00:38:39.680
SSH uh we want to see if there are

00:38:39.680 --> 00:38:42.760
interactions by these same Source IPS uh

00:38:42.760 --> 00:38:46.079
actually on the uh website that we're

00:38:46.079 --> 00:38:50.200
hosting uh so seeing a high number of

00:38:50.200 --> 00:38:53.400
failed values looking at actions also is

00:38:53.400 --> 00:38:55.599
a use case here for just bringing in

00:38:55.599 --> 00:38:57.680
that context and seeing if there's any

00:38:57.680 --> 00:39:00.520
sort of relationship between the data uh

00:39:00.520 --> 00:39:04.000
this is discussed often as correlation

00:39:04.000 --> 00:39:07.680
of logs I'm usually careful about using

00:39:07.680 --> 00:39:09.440
the term correlation in talking about

00:39:09.440 --> 00:39:11.119
spun queries especially in Enterprise

00:39:11.119 --> 00:39:12.640
security talking about correlation

00:39:12.640 --> 00:39:16.119
searches where I typically think of

00:39:16.119 --> 00:39:18.480
correlation searches as being

00:39:18.480 --> 00:39:20.599
overarching Concepts that cover data

00:39:20.599 --> 00:39:23.920
from multiple data sources and in this

00:39:23.920 --> 00:39:26.480
case correlating events would be looking

00:39:26.480 --> 00:39:28.400
at unique data types that are

00:39:28.400 --> 00:39:31.240
potentially related uh in finding that

00:39:31.240 --> 00:39:33.839
logical connection uh for the condition

00:39:33.839 --> 00:39:35.880
that's a little bit more up to the user

00:39:35.880 --> 00:39:38.319
it's not uh quite as easy as say

00:39:38.319 --> 00:39:41.520
pointing to a specific data

00:39:41.520 --> 00:39:44.880
model so we are going to look at one

00:39:44.880 --> 00:39:47.920
more sub search here and this case is

00:39:47.920 --> 00:39:52.240
going to apply uh the join command and

00:39:52.240 --> 00:39:55.680
so I talk about using lookup files uh or

00:39:55.680 --> 00:39:59.000
uh other data returned by sub searches

00:39:59.000 --> 00:40:01.599
uh to enrich to bring more data in

00:40:01.599 --> 00:40:05.599
rather than filter um we are going to

00:40:05.599 --> 00:40:08.960
look at our first part of the command

00:40:08.960 --> 00:40:11.480
here uh and this is actually just a

00:40:11.480 --> 00:40:15.720
simple uh stats report based on this rex

00:40:15.720 --> 00:40:18.079
that keeps coming through uh the SPL to

00:40:18.079 --> 00:40:21.000
give us those user and Source IP Fields

00:40:21.000 --> 00:40:24.079
uh so our result here is authentication

00:40:24.079 --> 00:40:26.200
failures for all these web hosts so

00:40:26.200 --> 00:40:28.760
similar to what we had previously

00:40:28.760 --> 00:40:31.200
returned and then we're going to take a

00:40:31.200 --> 00:40:33.319
look at the results of the sub search

00:40:33.319 --> 00:40:35.400
here actually split this up so that we

00:40:35.400 --> 00:40:38.839
can see uh the first two lines we're

00:40:38.839 --> 00:40:41.760
looking at our web access logs for

00:40:41.760 --> 00:40:45.560
purchase actions uh and then we are

00:40:45.560 --> 00:40:50.599
looking at uh our stats count for errors

00:40:50.599 --> 00:40:52.960
and stats count for successes we have

00:40:52.960 --> 00:40:55.079
pretty limited status code to return in

00:40:55.079 --> 00:40:59.240
this data so this uh is is uh viable for

00:40:59.240 --> 00:41:01.800
the data present uh to observe our

00:41:01.800 --> 00:41:03.280
errors and

00:41:03.280 --> 00:41:05.880
successes and then we are actually

00:41:05.880 --> 00:41:08.160
creating a new field based on the

00:41:08.160 --> 00:41:10.839
statistics that we're generating uh

00:41:10.839 --> 00:41:13.920
looking at our transaction errors so

00:41:13.920 --> 00:41:18.000
where we have uh high or low numbers uh

00:41:18.000 --> 00:41:22.079
of failed purchase actions uh and then

00:41:22.079 --> 00:41:25.599
summarizing that so in the case of our

00:41:25.599 --> 00:41:27.800
final command here another transforming

00:41:27.800 --> 00:41:30.640
command of table just to reduce this to

00:41:30.640 --> 00:41:35.079
a small data set uh to use in the subur

00:41:35.079 --> 00:41:37.440
and so in this case we have our host

00:41:37.440 --> 00:41:39.400
value and then our transaction error

00:41:39.400 --> 00:41:41.480
rate that we observe from the web access

00:41:41.480 --> 00:41:44.760
logs and then over in our other search

00:41:44.760 --> 00:41:48.640
here uh we are going to perform a left

00:41:48.640 --> 00:41:51.400
join based on this host field so you see

00:41:51.400 --> 00:41:53.359
in our secure logs we still have the

00:41:53.359 --> 00:41:55.800
same host value and this is going to be

00:41:55.800 --> 00:41:59.640
used uh to to actually add our

00:41:59.640 --> 00:42:02.760
transaction uh error rates in for each

00:42:02.760 --> 00:42:06.400
host so as we observe uh increased

00:42:06.400 --> 00:42:08.640
authentication failures if there's a

00:42:08.640 --> 00:42:11.960
scenario for a breach and some sort of

00:42:11.960 --> 00:42:14.960
interruption to the ability to serve out

00:42:14.960 --> 00:42:17.520
or perform these purchase actions that

00:42:17.520 --> 00:42:20.960
that are affecting uh the intended

00:42:20.960 --> 00:42:23.200
operations of the web servers uh we can

00:42:23.200 --> 00:42:25.280
see that here of course our tutorial

00:42:25.280 --> 00:42:27.319
data there's not really much that

00:42:27.319 --> 00:42:29.880
jumping out or showing uh that there is

00:42:29.880 --> 00:42:32.599
any correlation between the two but the

00:42:32.599 --> 00:42:34.640
purpose of the join is to bring in that

00:42:34.640 --> 00:42:37.440
extra data set to give the context to

00:42:37.440 --> 00:42:39.839
further

00:42:41.040 --> 00:42:47.440
investigate so um that is uh the final

00:42:47.440 --> 00:42:52.359
portion of the SPL demo uh and I do want

00:42:52.359 --> 00:42:54.920
to say for any questions I'm going to

00:42:54.920 --> 00:42:56.960
take a look at the chat I'll do my best

00:42:56.960 --> 00:43:00.079
to answer any questions um and then if

00:43:00.079 --> 00:43:03.079
you have any other questions uh please

00:43:03.079 --> 00:43:05.800
feel free to reach out to my team at

00:43:05.800 --> 00:43:08.599
support keny group.com and we'll be

00:43:08.599 --> 00:43:11.920
happy to get back to you and help um I

00:43:11.920 --> 00:43:15.440
am taking a look

00:43:26.119 --> 00:43:29.119
through

00:43:32.200 --> 00:43:33.760
okay seeing some questions on

00:43:33.760 --> 00:43:38.280
performance of the uh Rex said Rex

00:43:38.280 --> 00:43:41.599
commands um so off the top of my head I

00:43:41.599 --> 00:43:43.800
I'm not sure about a direct performance

00:43:43.800 --> 00:43:46.400
comparison uh of the individual commands

00:43:46.400 --> 00:43:49.200
definitely want to look into that um and

00:43:49.200 --> 00:43:52.280
definitely follow up uh if you'd like to

00:43:52.280 --> 00:43:54.280
uh explain a more detailed scenario or

00:43:54.280 --> 00:43:57.119
look at some SPL uh that we can apply in

00:43:57.119 --> 00:43:58.400
observe those

00:43:58.400 --> 00:44:01.680
changes um the question on getting the

00:44:01.680 --> 00:44:05.480
data set uh that is what I mentioned at

00:44:05.480 --> 00:44:07.520
the beginning uh reach out to us for the

00:44:07.520 --> 00:44:10.119
slides uh or just uh reach out about the

00:44:10.119 --> 00:44:15.480
link and the uh Splunk tutorial data you

00:44:15.480 --> 00:44:17.880
can actually search that as well um and

00:44:17.880 --> 00:44:20.400
there's documentation on how to use the

00:44:20.400 --> 00:44:22.400
tutorial data one of the first links

00:44:22.400 --> 00:44:25.640
there uh takes you to a page that has uh

00:44:25.640 --> 00:44:29.079
it is a tutorial data zip file uh and

00:44:29.079 --> 00:44:31.079
instructions on how to injust that it's

00:44:31.079 --> 00:44:34.079
just an upload uh for your specific

00:44:34.079 --> 00:44:37.599
environment so uh in add data and then

00:44:37.599 --> 00:44:40.040
upload data two clicks uh and upload

00:44:40.040 --> 00:44:43.400
your file so that is uh freely available

00:44:43.400 --> 00:44:45.760
for anyone uh and again that package is

00:44:45.760 --> 00:44:47.440
dynamically updated as well so your time

00:44:47.440 --> 00:44:51.359
stamps are pretty close to to normal uh

00:44:51.359 --> 00:44:53.440
as you download the app kind of depends

00:44:53.440 --> 00:44:55.920
on the time of the the cycle for the

00:44:55.920 --> 00:44:58.559
update um but search overall time you

00:44:58.559 --> 00:45:02.359
won't have any issues there um and then

00:45:02.359 --> 00:45:05.119
yeah again on receiving slides uh reach

00:45:05.119 --> 00:45:08.240
out to my team uh and we're happy to to

00:45:08.240 --> 00:45:10.240
provide those discuss further and we'll

00:45:10.240 --> 00:45:16.040
have uh the um the recording available

00:45:16.040 --> 00:45:18.400
for this session you should be able to

00:45:18.400 --> 00:45:20.680
after uh the recording processes when

00:45:20.680 --> 00:45:22.880
the session ends uh actually use the

00:45:22.880 --> 00:45:24.640
same link and you can watch this

00:45:24.640 --> 00:45:26.480
reporting and post uh without having to

00:45:26.480 --> 00:45:31.800
sign up or transfer that file so

00:45:33.680 --> 00:45:38.319
um so okay Chris seeing uh seeing your

00:45:38.319 --> 00:45:41.240
comment there um let me know if you want

00:45:41.240 --> 00:45:44.480
to reach out to me directly anyone as

00:45:44.480 --> 00:45:49.440
well um we can discuss what slides and

00:45:49.440 --> 00:45:51.640
presentation you had attended I'm not

00:45:51.640 --> 00:45:55.359
sure I have the attendance report uh for

00:45:55.359 --> 00:45:57.319
for what You' seen previously so uh

00:45:57.319 --> 00:46:00.240
happy to get those for

00:46:06.720 --> 00:46:10.319
you all right and uh seeing thanks Brett

00:46:10.319 --> 00:46:13.079
so you see Brett Woodruff in the chat

00:46:13.079 --> 00:46:16.680
commenting uh systems engineer on the uh

00:46:16.680 --> 00:46:18.640
expertise on demand team so very

00:46:18.640 --> 00:46:20.400
knowledgeable guy and he's going to be

00:46:20.400 --> 00:46:23.720
presenting next month's session uh that

00:46:23.720 --> 00:46:25.400
is going to take this concept that we

00:46:25.400 --> 00:46:28.760
talked about the subur in as a just

00:46:28.760 --> 00:46:30.760
general search topic he's going to go

00:46:30.760 --> 00:46:34.319
specifically into Data enrichment using

00:46:34.319 --> 00:46:38.079
uh joins lookup commands and how we see

00:46:38.079 --> 00:46:41.079
uh that used in the wild so definitely

00:46:41.079 --> 00:46:43.359
excited for that one encourage you to

00:46:43.359 --> 00:46:46.480
register for for that

00:46:46.920 --> 00:46:52.240
event all right I'm not seeing any more

00:46:55.839 --> 00:46:57.800
questions

00:46:57.800 --> 00:47:02.119
all right with that uh I am stopping my

00:47:02.119 --> 00:47:05.079
share I'm going to hang around for a few

00:47:05.079 --> 00:47:07.440
minutes uh but thank you all for

00:47:07.440 --> 00:47:11.079
attending and we'll see you on the next

00:47:15.920 --> 00:47:18.920
session