Neural Networks
One neuron draws one straight line. A neural network joins many neurons in layers, so it can draw bent shapes. In this lesson you do a forward pass by hand and write it with NumPy matrices. You also count the parameters and solve XOR.
New words
- Layer
- A group of neurons that work side by side at one step.
- Hidden layer
- A layer between the input and the output.
- Forward pass
- Sending one input through all the layers to get an output.
- Activation function
- A rule applied to each neuron that lets the network bend its line.
- Parameters
- All the weights and biases of the network.
- Width and depth
- Width is neurons per layer. Depth is the number of layers.
- Loss
- One number that shows how wrong the network is.
What you will learn
- What if one line is not enough?
- Layers: input, hidden, output
- A forward pass by hand
- A layer is a matrix times a vector
- Why do hidden layers bend the line? Solving XOR
- Width, depth and counting parameters
- The loss: how wrong is the network?
- Try it: a 2-3-1 network
- Common mistakes
- Explain it to a friend
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.