Underfitting occurs when a machine learning model is too simplistic to capture the underlying patterns in the data, resulting in poor performance on both training and test sets. It fails to learn the signal from the data, often due to insufficient model complexity, inadequate features, or insufficient training.
Key Characteristics:
- High Bias, Low Variance: The model makes overly simplistic assumptions, leading to consistently inaccurate predictions.
- Poor Training Accuracy: Unlike overfitting, underfitting performs poorly even on the training data.
- Caused by Oversimplification: Common in models that are too shallow, use too few parameters, or are trained on insufficient features or epochs.
- Flat Learning Curve: Both training and validation losses remain high and closely aligned.
- Correctable: Can often be resolved by increasing model complexity, feature engineering, or extending training.
Applications (and Risk Areas):
- Baseline Models: Simple models used as a starting point often underfit by design.
- Real-Time Systems: Conservative models may intentionally underfit to prioritize speed or resource constraints.
- Early Training Stages: Models may appear underfitted before sufficient training iterations.
- Complex Domains: Domains like natural language processing or computer vision require higher-capacity models to avoid underfitting.
Why It Matters:
Underfitting results in low accuracy and missed insights, making it unsuitable for practical deployment. Avoiding underfitting is essential for creating models that can learn meaningful patterns and deliver value in real-world use cases.
