10.1: Introduction to Neural Networks - The Nature of Code

Welcome to Chapter 10 of The Nature of Code (http://natureofcode.com/): Neural Networks.

In this video, I provide a brief introduction neural networks and an overview of topics in upcoming videos.

Next video (Perceptron): https://youtu.be/ntKn5TPHHAk

This video is also part of session 4 of my Spring 2017 ITP "Intelligence and Learning" course (https://github.com/shiffman/NOC-S17-2-Intelligence-Learning/tree/master/week4-neural-networks)

Support this channel on Patreon: https://patreon.com/codingtrain
To buy Coding Train merchandise: https://codingtrain.storenvy.com
To donate to the Processing Foundation: https://processingfoundation.org/

Send me your questions and coding challenges!: https://github.com/CodingTrain/Rainbow-Topics

Contact:
Twitter: https://twitter.com/shiffman
The Coding Train website: http://thecodingtrain.com/

Links discussed in this video:
The Nature of Code: http://natureofcode.com/
Session 4 of Intelligence and Learning: https://github.com/shiffman/NOC-S17-2-Intelligence-Learning/tree/master/week4-neural-networks
Perceptron on Wikipedia: https://en.wikipedia.org/wiki/Perceptron
My Simple Artificial Neural Network JavaScript Library: https://github.com/shiffman/Neural-Network-p5

Books discussed in this video:
Tariq Rashid's Make Your Own Neural Network: https://www.amazon.com/Make-Your-Own-Neural-Network-ebook/dp/B01EER4Z4G
Marvin Minsky's Perceptrons: https://www.amazon.com/Perceptrons-Introduction-Computational-Marvin-Minsky/dp/0262631113

Source Code for the all Video Lessons: https://github.com/CodingTrain/Rainbow-Code

p5.js: https://p5js.org/
Processing: https://processing.org

The Nature of Code playlist: https://www.youtube.com/user/shiffman/playlists?view_as=subscriber&shelf_id=6&view=50&sort=dd
For More Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
For More Intelligence and Learning: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6YJ3XfHhT2Mm4Y5I99nrIKX

Hello, welcome to a video that
at this present time doesn't exist,

but when you are watching this video,

right there, to the right of 9: Genetic Algorithms

will be the number 10,

and it will say next to that, Neural Networks.

So I am embarking on a journey
to learn about neural networks:

what they are, how you program them,

what's the kind of math
and stuff you need to know

to make them work, and then what kinds of creative

and experimental outcomes can you have.

Now, it should be said that there are lots

and lots of machine learning libraries out there.

There are lots of examples
and resources for doing this.

I want to--

Hold on, I'm still talking.

I don't know where--
Where did I put that book?

Ah, it's over here.

I want to reference this book,

Make Your Own Neural Network by Tariq Rashid,

which I used to develop a lot of the materials

that I will be presenting to you

and developing during this series of videos.

And I should also say that, you know--

And this book, this book has all sorts--

has how to program your own neural network

from scratch and without even knowing anything

about programming in Python,

because, as I might have said earlier today,

any reasonable person would start
and make a video tutorial series

about programming a neural network
from scratch in Python,

but I don't really--

I'm not very reasonable or logical,

and I just constantly make mistakes with everything.

And here's a mistake that I'm going to make.

I'm going to do all this in JavaScript.

And the reason for doing that

is to have everything run in the browser,

on the web, and also really for me
to learn about how to do this stuff.

So I am going to build a set--

I'm going to build a simple
neural network library in JavaScript,

not to make something efficient,
not to make something robust,

but to learn about the mechanics
of how all this stuff works,

because, ultimately--
and you might want to

just enjoy your summer, or maybe
you're watching this during the winter,

and get outside and do something else

and not watch these videos
and just skip ahead to later,

because I'm going to do a bunch
of coding challenges and projects

that involve that neural network library
and also other neural network libraries,

namely something called TensorFlow, in future videos.

But these first videos of building
the neural network library,

which I will do over a series,

really just for me to learn
how to do this stuff.

And if you want to watch and sort of

give me some good feedback
and see if you can follow along

and improve on what I'm doing

and help me with it,
that would be great.

So, okay, what-- what--

Hello, am I just rambling here?

I am. But why are we here?

So I'm going to go--

So the Nature of Code materials in this video

sits in the Nature of Code playlist

is all about looking at things in nature,

in our physical world,

and trying to unpack those things

and understand the algorithms behind those things,

and see if we can convert those things,

those algorithms, into code--

Ah, this is like going--
it's like autoplaying. (laughs)

How do I stop that?

..and turning those things into software
to make animations and creative projects.

Why not look at something really
interesting in nature: the brain?

So this is kind of a loose diagram of this idea

of an actual biological neural network.

Apparently, I have one here--

I'm struggling quite a bit these days--

where there are these entities called neurons,

and they're connected to other neurons.

And there's a lot of, you know, mystery to this

and a lot of recent research [inaudible] neuroscience.

What I am focused on in this series of videos

is what kinds of computational systems can be built,

inspired by the actual biological neural network,

biological brain, and made into something

called an artificial neural network?

And what kinds of applications
and outcomes can we create?

So what is the analog?

What is the neuron in our code?

How does it receive inputs?
How does it generate outputs?

So my brain does this.

It receives all these inputs
from light in the room

that travel through my retina
and into the brain and the signals

then produce outputs that allow me
to catch something or read some words.

How can that process be simulated in software?

And what types of outcomes can we generate?

And the very first thing that I'm going to do

is look at the simplest possible neural network,

a net--
It's not even a network at all.

It has one neuron, a processor neuron,

that receives two inputs and generates an output.

And that's called a perceptron.

So if you look at the next videos in this playlist,

I am going to build, in Processing,

a perceptron example just to show the mechanics

of how this works and to produce some trivial example

that doesn't necessarily have a very powerful outcome,

but gives us--
Because if we can build

and understand how this single neuron

receives inputs, processes those,
and generates an output,

then we can start to connect those together

to create more sophisticated systems

that can begin to generate outputs based on--

more complex outputs based on more complex inputs.

And this is kind of--

sits right there in the world of machine learning,

this idea of I have some data
that I want to make sense of.

That data is an input to a machine learning algorithm.

That algorithm is going to generate an output.

So maybe the data is an image.

The machine learning algorithm
is going to guess is it a cat or a dog?

Or maybe that input is the specs of the house,

you know, square footage, number of bedrooms,

etc., etc., and this machine learning system

is going to generate an output: a predicted price.

So there are lots of other machine learning algorithms

besides just neural network based ones,

and I do have another videos series
that cover some of those,

but ultimately I want to learn
how a neural network works,

so I can place it right there
and start to make sense of data,

and generate outputs from it.

So if you want to continue along,

the way this video series will work,

first there'll be a perceptron,
which is this thing.

Then I'm going to talk--
after the perceptron's done,

I'm going to talk about what the limitations

of the perceptrons are and why it is

that if we can create a multi-layered perceptron,

meaning many of these perceptrons
all connected to each other,

what we can start to build
and create afterwards.

So that's my rambling introduction

that apparently you just watched,

because-- I mean, maybe
no one will ever watch this,

but probably somebody will.

And I'll see you. Follow along.

I look forward to your feedback.
I hope this goes okay.

That's a pretty good goal.
Just okay is fine.

And I'll see you in these
future videos as I keep going.

Thanks for watching.

(upbeat music)