What is Training?
Training is the process of teaching a neural network to perform a task by exposing it to examples and adjusting its parameters based on errors.
Training Phases
The stages of training a neural network.
Initialization
Set random starting weights. Good initialization helps training.
Forward Pass
Input flows through the network to produce predictions.
Loss Calculation
Compare predictions to ground truth with a loss function.
Backpropagation
Compute gradients of loss with respect to each weight.
Optimization
Update weights using gradient descent.
Key Concepts
Epoch
One complete pass through the entire training dataset.
Batch Size
Number of examples processed before updating weights.
Overfitting
Model memorizes training data but fails on new data.
Regularization
Techniques to prevent overfitting (dropout, weight decay).
Training Progress Visualizer
Watch a network learn in real-time
Training Progress
Watch a neural network learn
Epoch
Training Loss
Validation Loss
Accuracy
Loss Over Time
Accuracy Over Time
Watch for the gap between training and validation loss. When validation loss starts increasing while training loss decreases, the model is overfitting—memorizing training data instead of learning generalizable patterns.
Key Takeaways
- 1Training iteratively reduces prediction errors
- 2Overfitting is the main enemy—always validate on held-out data
- 3Batch size and learning rate significantly affect training
- 4Modern LLMs require massive compute for training