About standardization

#1

I want to know why we have to standardise the given data before processing it.

#2

Hey @light007, let me explain you that from scratch :

Standardization and Normalization are two major techniques of feature scaling.

Gradient Descent Based Algorithms

Machine learning algorithms like linear regression, logistic regression, neural network, etc. that use gradient descent as an optimization technique require data to be scaled. Take a look at the formula for gradient descent below:
image
The presence of feature value X in the formula will affect the step size of the gradient descent. The difference in ranges of features will cause different step sizes for each feature. To ensure that the gradient descent moves smoothly towards the minima and that the steps for gradient descent are updated at the same rate for all the features, we scale the data before feeding it to the model.

Distance-Based Algorithms

Distance algorithms like KNN, K-means and SVM are most affected by the range of features. This is because behind the scenes they are using distances between data points to determine their similarity. Therefore, we scale our data before employing a distance based algorithm so that all the features contribute equally to the result.

Normalization is a scaling technique in which values are shifted and rescaled so that they end up ranging between 0 and 1. It is also known as Min-Max scaling.

Here’s the formula for normalization:

image

Standardization is another scaling technique where the values are centered around the mean with a unit standard deviation. This means that the mean of the attribute becomes zero and the resultant distribution has a unit standard deviation.

Here’s the formula for standardization:
image

Hope this helps.
Happy Learning :slight_smile: