Feature Scaling in Machine Learning: Standardization vs Normalization
Feature Scaling in Machine Learning: Standardization vs Normalization (Easy Explanation)
Feature scaling is one of the most important preprocessing steps in machine learning. Many algorithms work better when all features are on a similar scale. If the data is not scaled properly, some features may dominate others just because their values are larger, not because they are more important.
In this blog, we will understand what feature scaling is, why it is needed, and the difference between Standardization and Normalization with simple examples.
What is Feature Scaling?
Feature scaling means changing the range of data so that all features have similar values.
It does not change the meaning of data; it only adjusts the scale.
For example:
If one column has values like 2, 5, 8
and another has 200, 500, 800,
the second feature may dominate the model. Scaling avoids this problem.
Why Feature Scaling Is Needed?
Most ML algorithms use distance or gradient-based calculations. These methods get affected when features have very different numerical ranges.
Algorithms that require scaling
- K-Nearest Neighbors (KNN)
- Support Vector Machine (SVM)
- Linear Regression
- Logistic Regression
- Gradient Descent based models
- PCA (Principal Component Analysis)
- Neural Networks
Algorithms that do NOT require scaling
- Decision Trees
- Random Forest
- XGBoost / LightGBM
- Naive Bayes
These models do not rely on distance or gradient magnitude.
Standardization vs Normalization
Feature scaling is mainly done in two ways:
1. Standardization (Z-score Scaling)
2. Normalization (Min-Max Scaling)
Both methods aim to scale values, but the logic behind them is different.
1. Standardization (Z-Score Scaling)
Standardization transforms the data so that:
Mean becomes 0
Standard deviation becomes 1
Formula
z = x-u/sigma
Where:
x= original value
u = mean of the feature
signa= standard deviation
After applying the formula, values generally lie between -3 to +3.
When to use Standardization
- When your data contains outliers
- When you don’t want data to be forced into a specific range
- Works best for SVM, Logistic Regression, Linear Regression
Simple Example
Suppose the height column contains values:
160, 170, 180
Mean = 170
Standard deviation = approx 8.16
For 180:
z = 180-170/8.16= 1.22
This means the height is 1.22 standard deviations above the mean.
Code :-
from sklearn.preprocessing import StandardScaler
import numpy as np
data = np.array([[10], [20], [30], [40], [50]]
scaler = StandardScaler()
scaled_data = scaler.fit_transform(data)
print(scaled_data)
2. Normalization (Min-Max Scaling)
Normalization transforms data into a fixed range, usually 0 to 1.
Formula
x = x - xmin / xmax - xmin
Where:
x = original value
xmin = smallest value
xmax = largest value
When to use Normalization
- When you need values strictly between 0 and 1
- When the dataset does not contain many outliers
- Commonly used for Neural Networks and KNN
Simple Example
Suppose marks are:
40, 60, 80
Min = 40
Max = 80
For 60:
x' = 60 - 40/80 - 40 = 0.5
Code :-
from sklearn.preprocessing import MinMaxScaler
import numpy as np
data = np.array([[2], [5], [10], [18]])
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(data)
print(scaled_data)
Difference Between Standardization and Normalization
Below is a clear summary of the key differences:
Standardization
- Does not force data into a range
- Uses mean and standard deviation
- Good for algorithms sensitive to outliers
- Range is usually -3 to +3
Normalization
- Forces values into 0–1
- Uses minimum and maximum
- Sensitive to outliers
- Simple and widely used
Which One Should You Use?
Choose Standardization when:
- There are outliers
- You are using SVM, logistic regression, linear regression
Choose Normalization when:
- A fixed 0–1 range is required
- Data has no major outliers
- You are applying deep learning models or KNN
Conclusion
Feature scaling is a crucial step in building accurate and stable machine learning models. Both Standardization and Normalization help bring all features to a similar scale but in different ways. Choosing the right method depends on your dataset and the ML algorithm you are using.
With proper scaling, your models will train faster, perform better, and give more reliable predictions.
#MachineLearning, #FeatureScaling, #Standardization, #Normalization, #DataScienceBasics, #MLPreprocessing
Comments
Post a Comment