0:00:00.259,0:00:03.421
What we hope to do with this meetup
0:00:03.851,0:00:10.620
is have something, given the spread of [br]the questionnaire results
0:00:10.620,0:00:12.681
we hope to do something which is kind of[br]
0:00:12.681,0:00:15.721
for people who don't know what[br]deep learning is
0:00:15.721,0:00:17.992
and want an introduction to TensorFlow
0:00:17.992,0:00:20.322
but also something which is more of a
0:00:20.322,0:00:24.021
like a crowd pleaser or something[br]which is more cutting edge
0:00:24.021,0:00:27.081
I am not going to say that this [br]thing is particularly cutting edge
0:00:27.081,0:00:31.553
because once we saw the responses,[br]we dialed things down a bit
0:00:31.553,0:00:37.803
But there will be more cutting edge stuff
0:00:37.803,0:00:42.811
and maybe we start to do other meetups[br]events in other formats
0:00:42.811,0:00:48.824
So it could be like we have [br]an experts' paper meeting
0:00:48.824,0:00:52.864
or we could split it now we can see [br]the size of people, size of the crowd
0:00:52.864,0:00:57.824
Anyway, let me talk a little bit about[br]going deeper with transfer learning
0:00:57.824,0:01:00.164
Unfortunately, this is something [br]some of you people
0:01:00.164,0:01:02.503
would have seen me do before
0:01:02.503,0:01:04.883
This is the first time I have [br]done it in tensorflow
0:01:05.122,0:01:07.272
and let me just explain that
0:01:07.272,0:01:10.162
Before, I have been programming this stuff
0:01:10.162,0:01:13.382
in Theano with the [br]Lasagna layers thing on top
0:01:13.382,0:01:19.431
and Theano is a research-based [br]deep learning framework, out of Montreal
0:01:19.431,0:01:22.683
but what I have concluded[br]since last summer
0:01:22.683,0:01:26.643
is that TensorFlow 's probably the winner[br]of this framework race
0:01:26.643,0:01:29.434
at least for foreseeable future
0:01:29.434,0:01:31.861
with all this nice industrial stuff
0:01:31.891,0:01:35.154
I should be retooling into TensorFlow
0:01:35.518,0:01:37.483
That's what I am taking the opportunity[br]to do for this
0:01:40.951,0:01:43.067
So, about me, sorry here we go
0:01:43.067,0:01:45.678
I have come up through finance, [br]startups and stuff
0:01:45.678,0:01:49.649
I took a year out basically in 2014[br]just for fun
0:01:49.649,0:01:53.559
I have been doing serious kind of [br]natural language processing since then
0:02:00.909,0:02:04.629
Basically, the overview for this[br]something more challenging talk
0:02:04.629,0:02:08.669
which will probably be 20 mins, 30 mins[br]depending on how it goes
0:02:08.669,0:02:13.889
I want to take a state-of-the-art[br]TensorFlow model
0:02:13.889,0:02:16.769
I want to solve a problem that [br]it wasn't trained for
0:02:16.769,0:02:20.928
And I am going to be using [br]deep learning as a component
0:02:20.928,0:02:25.960
of my solution rather than the [br]primary focus of what I am trying to build
0:02:25.960,0:02:32.900
So this is, in a way more of an industrial[br]or commercial kind of application
0:02:32.912,0:02:35.190
for what's going on here
0:02:35.190,0:02:38.510
So the goal for this kind of problem is
0:02:38.510,0:02:42.530
I want to distinguish pictures[br]of classic and modern sports cars
0:02:42.530,0:02:47.051
you will see some pictures of [br]classic and modern cars a bit later
0:02:48.433,0:02:51.722
It's not that easy to say what[br]the difference is
0:02:51.722,0:02:55.211
obviously, it could be [br]different types of images
0:02:55.211,0:02:57.454
and it could be lots of [br]different classes
0:02:57.454,0:03:00.992
I am just doing a very simple [br]two class thing
0:03:00.992,0:03:03.145
but it's complicated images
0:03:03.145,0:03:04.824
what I want to do is
0:03:04.824,0:03:06.114
I want to have a very small training time
0:03:06.114,0:03:08.381
so I don't want to be retraining [br]some huge network
0:03:08.381,0:03:12.895
Particularly, I have only got [br]in this case, 20 training examples
0:03:12.895,0:03:18.195
So I am not gonna do any fantastic[br]million image training
0:03:18.195,0:03:20.863
I have got 20 images with me
0:03:20.863,0:03:24.705
and I also want to be able to[br]put this in production
0:03:24.705,0:03:30.118
so I can just run it as a component of [br]something else
0:03:30.118,0:03:36.395
Basically, one of the things that is [br]carrying the deep learning world forward
0:03:36.395,0:03:40.196
is an image classification task [br]called ImageNet
0:03:40.196,0:03:42.406
this has been a competition where
0:03:42.406,0:03:47.407
they have 15 million labeled images[br]from 22,000 categories
0:03:47.407,0:03:49.858
and you can see some of them here
0:03:49.858,0:03:55.817
if we go for this.[br]this is a picture of a hotdog in a bun
0:03:55.817,0:03:57.786
and here are some of the categories
0:03:57.786,0:04:02.538
which will be some food I don't know
0:04:02.538,0:04:06.107
these are hotdogs, lots of [br]different pictures of hotdogs
0:04:06.107,0:04:09.058
lots of different pictures of cheeseburgers
0:04:09.058,0:04:11.848
lots of different pictures of plates
0:04:11.848,0:04:15.338
so the task for ImageNet is to classify
0:04:15.338,0:04:18.267
for any given, any one of these images
0:04:18.267,0:04:20.447
which of a thousand different [br]categories it is from
0:04:20.447,0:04:25.328
and it used to be that people could[br]score adequately well
0:04:25.328,0:04:28.558
and were making incremental changes in
0:04:28.558,0:04:30.558
how well they can do this
0:04:30.558,0:04:32.998
but the deep learning people came along
0:04:32.998,0:04:35.488
and kind of tore this to shreds
0:04:35.488,0:04:40.149
and Google came up with GoogLeNet
0:04:40.149,0:04:43.909
what we are actually going to use here,[br]back in 2014
0:04:43.909,0:04:49.649
suddenly, this stuff is now being done[br]with further iterations
0:04:49.649,0:04:52.808
of this kind of thing,[br]better than humans can
0:04:52.808,0:04:56.795
So the way you can measure whether[br]someone is better than humans
0:04:56.795,0:04:59.069
is, you take a human and see[br]whether it beats him
0:04:59.069,0:05:01.560
the question there is [br]are there labeling errors
0:05:01.560,0:05:03.720
there you need a committee of humans
0:05:03.720,0:05:06.250
so the way they label these things is
0:05:06.250,0:05:08.740
by running it on Mechanical Turk and
0:05:08.740,0:05:12.490
asking people what category is this[br]cheeseburger in
0:05:14.820,0:05:16.380
The network we are going to use here
0:05:16.380,0:05:23.421
is the 2014 state-of-the-art GoogLeNet,[br]also called Inception version 1
0:05:23.421,0:05:25.690
The nice thing about this is that
0:05:25.690,0:05:30.942
there is an existing model[br]already trained for this task
0:05:30.942,0:05:33.772
and it's available for download[br]it's all free
0:05:33.772,0:05:38.952
and there are lots of different [br]models out there
0:05:38.952,0:05:41.362
there's a model zoo for TensorFlow
0:05:41.362,0:05:44.351
So, what I have on my machine
0:05:44.351,0:05:48.531
and this is a small model, [br]it's a 20 megabytes kind of model
0:05:48.531,0:05:50.276
So it is not a very big model
0:05:50.276,0:05:57.291
Inception 4 is a 200 MB kind of model[br]which is a bit heavy
0:05:57.291,0:05:59.423
I am working here on my laptop
0:05:59.423,0:06:01.212
you are gonna see it working in real-time
0:06:01.212,0:06:07.254
and the trick here is instead of [br]a softmax layer at the end
0:06:07.254,0:06:12.984
I will show you the diagram, it should be [br]clear to anyone who's following along
0:06:12.984,0:06:19.082
instead of using logits to get me [br]the probablilities
0:06:19.082,0:06:21.133
I am going to strip that away
0:06:21.133,0:06:23.074
and I am going to train [br]a support vector machine
0:06:23.074,0:06:24.884
to distinguish these classes
0:06:24.884,0:06:29.854
I am not going to retrain the [br]Inception network at all
0:06:29.854,0:06:32.474
I am going to just use it as a component
0:06:32.474,0:06:34.913
strip off the top classification piece
0:06:34.913,0:06:38.234
and replace it with an SVM
0:06:38.234,0:06:40.384
Now, SVMs are pretty well understood
0:06:40.384,0:06:44.624
here I am just using Inception [br]as a featurizer for images
0:06:44.624,0:06:47.285
So here's a network picture
0:06:47.285,0:06:52.015
Basically, this is what the ImageNet [br]network is designed for
0:06:52.015,0:06:54.334
you put in an image at the bottom
0:06:54.334,0:06:57.445
there is this black box which is the [br]Inception network
0:06:57.445,0:07:00.745
which is a bunch of CNNs or [br]convolutional neural networks
0:07:00.745,0:07:02.596
followed by Dense network
0:07:02.596,0:07:04.846
followed by these logits
0:07:04.846,0:07:07.976
and these logits layers is essentially [br]the same as the 0 to 10
0:07:07.976,0:07:17.037
that Sam had for his digits, 1 to 1000 [br]for the different classes for ImageNet
0:07:17.037,0:07:20.418
To actually get the ImageNet output
0:07:20.418,0:07:27.387
it uses a softmax function and [br]then chooses the highest one of these
0:07:27.387,0:07:28.908
to give you this is the class[br]that this is in
0:07:28.908,0:07:32.167
What I am going to do is [br]I am going to ignore this
0:07:32.167,0:07:35.337
neat piece of classification technology[br]that they have got
0:07:35.337,0:07:44.148
let's say we use these outputs as inputs[br]to SVM and just treat these as features
0:07:44.148,0:07:46.567
Now if we pick out one of these
0:07:46.567,0:07:50.698
this class could be cheeseburger[br]and this class could be parrot
0:07:50.698,0:07:54.067
and this other class could be Husky dog
0:07:54.067,0:07:57.178
there is all sorts of classes in here
0:07:57.178,0:07:59.709
but basically what I will be doing is that
0:07:59.709,0:08:02.248
I will be extracting out the features [br]of these photos
0:08:02.248,0:08:04.948
saying how much of this photo [br]is like a parrot
0:08:04.948,0:08:08.938
how much of this is like a Husky dog
0:08:08.938,0:08:13.229
Now it turns out that modern cars and [br]classic cars can be distinguised that way
0:08:13.229,0:08:18.659
Let me go to some code
0:08:18.659,0:08:20.600
Ok this code is all up on GitHub
0:08:30.950,0:08:34.300
Can everyone see this enough
0:08:38.380,0:08:42.230
So basically, I am pulling in TensorFlow
0:08:45.400,0:08:49.251
I pull in this model
0:08:49.251,0:08:52.780
Here is what the Inception architecture is
0:08:52.780,0:08:56.971
It feeds forward this way,[br]here you put your image
0:08:56.971,0:08:59.901
it goes through lots and lots of [br]convolutional layers
0:08:59.901,0:09:03.490
all the way up to the end [br]with softmax and the output
0:09:03.490,0:09:06.922
So having done that, what I will do is
0:09:06.922,0:09:09.741
actually I have a download [br]for the checkpoint
0:09:09.741,0:09:16.562
this is the checkpoint here which [br]is a tar file, I have it locally stored
0:09:16.562,0:09:18.500
It doesn't download it now
0:09:18.500,0:09:25.262
but it is all there, even the [br]big models are there up from Google
0:09:25.262,0:09:27.762
so they have retrained these
0:09:27.762,0:09:30.483
so the Inception thing takes about a week
0:09:30.483,0:09:33.792
to retrain on a bunch of, [br]it could be 64 GPUs
0:09:33.792,0:09:36.864
so you don't really want to be[br]training this thing on your own
0:09:36.864,0:09:40.793
you also need the ImageNet training set[br]
0:09:40.793,0:09:48.384
it is a 140 GB file [br]which is no fun to download
0:09:50.824,0:09:57.185
what I am doing here is basically[br]there is also an Inception library
0:09:57.185,0:10:04.043
which is part of the TF-Slim [br]this thing is desinged such that
0:10:04.043,0:10:08.264
it already knows the network[br]it can preload it
0:10:08.264,0:10:12.290
this has loaded it, [br]I can get some labels
0:10:12.290,0:10:17.184
This is loading up the ImageNet labels
0:10:17.184,0:10:25.565
I need to know which location [br]corresponds to which class like the digits
0:10:31.285,0:10:33.305
Here we are going through [br]basically the same steps
0:10:33.305,0:10:39.068
as the MNIST example in that[br]we reset the default graph
0:10:39.068,0:10:44.586
we create a placeholder which is [br]where my images are going to go
0:10:44.586,0:10:47.575
this is as an input [br]but from this image input
0:10:47.575,0:10:49.904
I am then going to do some TensorFlow steps
0:10:49.904,0:10:52.286
because TensorFlow [br]has various preprocessing
0:10:52.286,0:10:55.767
or graphics handling commands
0:10:55.767,0:10:57.747
because a lot of this stuff[br]works with images
0:10:57.747,0:11:02.547
so there's all sorts of clipping[br]and rotating stuff
0:11:02.547,0:11:04.778
so it can preprocess these images
0:11:04.778,0:11:08.485
I am also going to pull out a numpy image
0:11:08.485,0:11:10.828
so I can see what it is actually looking at
0:11:10.828,0:11:14.850
here with this Inception version 1
0:11:14.850,0:11:20.906
I am going to pull in the entire[br]Inception version 1 model
0:11:23.356,0:11:26.568
My net function rather than being [br]just picks and random weights
0:11:26.568,0:11:29.978
is gonna be assigned this [br]from this checkpoint
0:11:29.978,0:11:34.418
when I run the init thing from my graph
0:11:34.418,0:11:37.478
or in my session, it won't initialize[br]everything from random
0:11:37.478,0:11:39.479
it will initialize everything from disk
0:11:39.479,0:11:42.028
so this will define the model
0:11:42.028,0:11:45.358
and now let's proceed
0:11:45.358,0:11:51.609
one of the issues with having this [br]on a nice TensorFlow graph
0:11:51.609,0:11:56.658
is it just says input, Inception1, output
0:11:56.658,0:11:59.939
so there's a big block there [br]you can delve into it if you want
0:11:59.939,0:12:05.790
let me just show you[br]let's go back a bit
0:12:08.320,0:12:11.300
So this is the code [br]behind the Inception1 model
0:12:11.300,0:12:16.060
so this is actually smaller than the [br]Inception2 and Inception3
0:12:16.060,0:12:22.331
basically, we have a kind of a base[br]Inception piece, just this
0:12:22.331,0:12:24.971
and these are combined together
0:12:24.971,0:12:33.441
and this is a detailed model put together [br]by many smart people in 2014
0:12:33.441,0:12:35.472
it's got much more complicated since then
0:12:35.472,0:12:38.912
fortunately, they have written the code [br]and we don't have to
0:12:43.422,0:12:46.321
So here what I am gonna do is [br]I am gonna load an example image
0:12:46.321,0:12:50.581
just to show you[br]one of the the things here is
0:12:50.581,0:12:56.396
TensorFlow in order to become efficient[br]wants to do the loading itself
0:12:56.396,0:13:01.344
So in order to get this pumping [br]information through
0:13:01.344,0:13:03.633
it wants you to set up hues of images
0:13:03.633,0:13:10.263
it will then handle the whole ingestion[br]process itself
0:13:10.263,0:13:14.153
the problem with that is [br]it's kind of complicated to do
0:13:14.153,0:13:16.023
in a Jupyter notebook right here
0:13:16.023,0:13:19.133
so here I am going to do [br]the very simplest thing
0:13:19.133,0:13:22.393
which is load a numpy image [br]and stuff the numpy image in
0:13:22.393,0:13:24.883
but what TensorFlow would love me to do
0:13:24.883,0:13:29.413
is create , as you see in this one
0:13:29.413,0:13:34.024
create a file name queue and it will
0:13:34.024,0:13:35.314
then run the queue, do the matching
0:13:35.314,0:13:36.674
and do all of this stuff itself
0:13:36.674,0:13:41.093
because then it can lay it out across [br]potentially distributed cluster
0:13:41.093,0:13:43.414
and do everything just right
0:13:43.414,0:13:50.254
here I do kind of the simple read the image
0:13:50.254,0:13:59.507
so this image is a tensor [br]which is 224 by 224 by RGB
0:13:59.507,0:14:03.478
this is kind of sanity check[br]what kind of numbers I got in the corner
0:14:03.478,0:14:05.667
and then what I am gonna do is
0:14:05.667,0:14:08.016
i am going to crop out the [br]middle section of it
0:14:08.016,0:14:10.761
this happens to be the right size already
0:14:10.761,0:14:13.495
basically if you got odd shapes
0:14:13.495,0:14:15.136
you need to think about [br]how am I gonna do it
0:14:15.136,0:14:18.956
am I going to pad it[br]what do you do
0:14:18.956,0:14:21.947
because in order to make this efficient
0:14:21.947,0:14:29.056
TensorFlow wants to lay it out without [br]all this variability in image size
0:14:29.056,0:14:34.475
one set of parameters and it's then going[br]to blast it across your GPU
0:14:34.475,0:14:37.865
so let's just run this thing
0:14:37.865,0:14:39.697
so now we have defined the network
0:14:39.697,0:14:45.767
here I am going to pick a session[br]here I am going to init the session
0:14:45.767,0:14:47.839
it loads the data, and then I am going
0:14:47.839,0:14:52.037
to pick up the numpy image and the[br]probabilities from the top layer
0:14:52.037,0:14:54.677
I am just gonna show it
0:14:57.507,0:15:01.366
here is the image [br]this is image I pulled out of the disk
0:15:01.366,0:15:06.327
you can see here the probabilities, [br]the highest probability is Tabby cat
0:15:06.327,0:15:10.487
which is good, it's also interesting that
0:15:10.487,0:15:15.263
the second in line things are Tiger cat,[br]Egyptian cat, lynx
0:15:15.263,0:15:21.037
so it's got a fair idea that it is a cat[br]in particular, it is getting it right
0:15:21.037,0:15:26.169
ok so this is the same diagram [br]we have had before
0:15:26.169,0:15:32.729
what you have seen is this going in this [br]black box, coming out and telling us
0:15:32.729,0:15:35.868
the probabilities here, so what we are [br]now gonna do is
0:15:35.868,0:15:41.910
from the image to the black box and [br]just learn a bunch of features
0:15:50.030,0:15:52.720
let me just show you this on disk
0:16:11.300,0:16:13.304
so I have a cars directory here
0:16:13.957,0:16:17.848
and inside this thing,
0:16:24.238,0:16:25.788
I have surprisingly little data
0:16:36.648,0:16:39.863
In this directory, I just have a [br]bunch of car images
0:16:39.863,0:16:42.189
and I have two sets of images
0:16:42.189,0:16:47.659
one of which is called classic [br]and the other is called modern
0:16:47.659,0:16:52.010
so I basically I picked some[br]photos off Flickr
0:16:52.010,0:16:54.439
I put these into two separate directories
0:16:54.439,0:16:56.309
I am going to use those directory names
0:16:56.309,0:17:00.431
as the classification for these images
0:17:00.431,0:17:05.160
In the upper directory here[br]I got a bunch of test images
0:17:05.160,0:17:06.830
which I don't know the labels for
0:17:12.610,0:17:17.261
this picks out the list of classic , there[br]is a classic and a modern directory
0:17:17.261,0:17:21.990
I am gonna go through every file [br]in this directory
0:17:21.990,0:17:28.470
I am gonna crop it, I am gonna find [br]the logits level which is
0:17:28.470,0:17:33.441
all the classes and then I am just gonna [br]add these to features
0:17:33.441,0:17:36.601
So basically I am gonna do something[br]like a scikit-learn model
0:17:36.601,0:17:38.311
I am gonna fit SVM
0:17:38.311,0:17:42.111
so basically, this is featurizing [br]all these pictures
0:17:47.911,0:17:49.961
so here we go with the training data
0:17:55.571,0:17:56.972
here's some training
0:18:02.272,0:18:05.622
classic cars,[br]it went through the classic directory
0:18:05.622,0:18:08.782
modern cars, [br]it went through the modern directory
0:18:15.292,0:18:16.752
it's thinking hard
0:18:18.392,0:18:25.284
what I am gonna do now is [br]build SVM over those features
0:18:31.016,0:18:40.180
jump to 21:36
0:21:35.478,0:21:43.839
I restarted this thing
0:21:43.839,0:21:49.619
the actual training for this SVM thing[br]takes that long,
0:21:49.619,0:21:58.018
this is very quick, essentially 20 images [br]worth of a thousand features
0:21:58.018,0:22:01.840
so there was no big training loop to do
0:22:01.840,0:22:09.070
then I can run this on the actual models [br]in the directory, in the test set
0:22:09.070,0:22:12.680
so here this is images that it has never[br]seen before
0:22:12.680,0:22:16.440
it thinks that this is a modern car
0:22:16.440,0:22:19.020
this one it thinks is a classic car, [br]this one is classified as modern
0:22:19.020,0:22:26.301
so this is actually doing quite a good job[br]out of just 10 examples of each
0:22:26.301,0:22:32.770
it actually thinks this one is modern[br]it's not a sports car but anyway
0:22:32.770,0:22:38.939
so this is showing that the SVM we trained
0:22:38.939,0:22:42.901
can classify based on the features that[br]Inception is producing because
0:22:42.901,0:22:47.231
Inception understands "understands"[br]what images are about
0:22:47.231,0:22:50.801
so if I go back to here, [br]code is on GitHub
0:22:50.801,0:22:53.992
conclusions okay, this thing really works
0:22:53.992,0:22:58.402
we didn't have to train [br]a deep neural network
0:22:58.402,0:23:01.876
we could plug this TensorFlow model[br]into an existing pipeline
0:23:01.876,0:23:04.760
and this is actually something where
0:23:04.760,0:23:08.532
the TensorFlow Summit has something[br]to say about these pipelines
0:23:08.532,0:23:11.013
because not only are they talking[br]about deep learning
0:23:11.013,0:23:14.753
they are talking about the whole [br]cloud-based learning
0:23:14.753,0:23:19.453
and setting up proper processes
0:23:19.453,0:23:23.965
I guess, time for questions quickly
0:23:23.965,0:23:29.142
we can then do the [br]TensorFlow Summit wrap-up
0:23:33.212,0:23:37.144
"I am assuming that there is no [br]backpropagation here"
0:23:37.144,0:23:40.034
This includes no backpropagation
0:23:40.034,0:23:42.504
"End result is a feature"
0:23:45.884,0:23:53.135
I am just assuming that Inception,[br]you can imagine if the ImageNet thing
0:23:53.135,0:23:56.265
had focused more on products,[br]it could be even better
0:23:56.265,0:23:58.914
if it focused on man-made things
0:23:58.914,0:24:04.915
The ImageNet training set has an awful[br]lot of dogs in it, not that many cats
0:24:04.915,0:24:09.426
So, on the other hand it may be that [br]it has quite a lot of flowers
0:24:09.426,0:24:13.826
or maybe that it is saying I like this car[br]as modern car
0:24:13.826,0:24:16.046
because it's got petals for wheels
0:24:16.046,0:24:20.385
whereas the other one, the classic cars[br]tend to have round things for wheels
0:24:20.385,0:24:25.146
So it is abstractly doing this
0:24:25.146,0:24:29.918
It doesn't know about sports cars or [br]what they look like
0:24:29.918,0:24:31.587
But it does know about curves
0:24:34.607,0:24:37.527
"So for SVM, you don't use [br]TensorFlow anymore ?"
0:24:37.527,0:24:43.157
No, basically I have used TensorFlow to [br]create some features
0:24:43.157,0:24:45.308
Now, I don't want to throw it away
0:24:45.308,0:24:47.687
because hopefully I have got[br]a streaming process where
0:24:47.687,0:24:52.177
more and more images are chugged [br]through this thing
0:24:52.177,0:25:04.528
could not hear the question properly
0:25:07.058,0:25:10.068
There is an example code called[br]TensorFlow for poets
0:25:10.068,0:25:13.296
where they actually say that, [br]let's load up one of these networks
0:25:13.296,0:25:15.369
and then we will do some fine tuning
0:25:15.369,0:25:21.977
there you get involved in tuning [br]these neurons with some gradient descent
0:25:21.977,0:25:24.819
and you are taking some steps [br]and all this kind of thing
0:25:24.819,0:25:28.328
maybe you are having broad implications[br]across the whole network
0:25:28.328,0:25:32.819
which could be good if you have got[br]tons of data and tons of time
0:25:32.819,0:25:36.948
but this is a very simple way of just[br]tricking it to get it done
0:25:36.948,0:25:47.382
could not hear the comment properly
0:25:47.382,0:25:54.033
it will be a very small network[br]because SVM is essentially fairly shallow
0:25:54.033,0:26:06.532
could not hear the question
0:26:06.532,0:26:13.752
TensorFlow even though it has imported[br]this large Inception network
0:26:13.752,0:26:20.572
as far as I am concerned, [br]I am using a f(x) = y and that's it
0:26:20.572,0:26:25.062
but you can inquire what would it say[br]at this particular level
0:26:25.062,0:26:30.473
and these bunches of levels with various[br]component points along the way
0:26:30.473,0:26:33.654
I could take out other levels
0:26:33.654,0:26:35.783
I haven't tried it to have a look
0:26:35.783,0:26:40.083
There you get more like pictures[br]worth of features rather than
0:26:40.083,0:26:43.094
this string of a 1000 numbers
0:26:43.094,0:26:48.884
but each intermediate levels [br]will be pictures with CNN kind of features
0:26:48.884,0:26:53.544
on the other hand, if you want [br]to play around with this thing
0:26:53.544,0:26:57.654
there's this nice stuff called [br]the DeepDream kind of things
0:26:57.654,0:27:02.559
where they try and match images to [br]being interesting images
0:27:02.559,0:27:06.454
then you do the featurizing that looks at[br]different levels
0:27:06.454,0:27:12.415
the highest level is a cat but I want all[br]local features to be as fishy as possible
0:27:12.415,0:27:15.561
then you get like a fish-faced cat
0:27:15.561,0:27:20.010
that's the kind of thing you can do with[br]these kinds of features in models