- Feature transformation is simply a function that transforms features from one representation to another.
- Well there are many reasons, such as:
- Data types are not suitable to be fed into a machine learning algorithm, e.g. text, categories
- Feature values may cause problems during the learning process, e.g. data represented in different scales
- We want to reduce the number of features to plot and visualize data, speed up training or improve the accuracy of a specific model
- Linear Regression uses Gradient Descent for its calculation of Global Minima thats needs a even scaled data for better a prediction.
- Algorithms like KNN,K Means,Hierarichal Clustering use Eucledian Distance, Manhattan Distance etc. for the peorpose of mesure the distance between observations, They need scaled data too for the better prediction.
- Deep Learning Techniques needs Standardization, Scaling they are also Feature Transformation techniques.
- In practice, different types of variables are often encountered in the same data set.
- A significant issue is that the range of the variables may differ a lot.
- Using the original scale may put more weight on the variables with a large range.
- In order to deal with this problem, the technique of features rescaling need to be applied to independent variables or features of data in the step of data preprocessing.
- The terms "normalization" and "standardization" are sometimes used interchangeably, but they usually refer to different things.
- The goal of applying feature scaling is to make sure features are on almost the same scale so that each feature is equally important and make it easier to process by most machine-learning algorithms
- Some machine learning models are fundamentally based on distance matrix, also known as the distance-based classifier, for example, K-Nearest-Neighbours, SVM, and Neural Network.
- Feature scaling is extremely essential to those models, especially when the range of the features is very different.
- Otherwise, features with a large range will have a large influence in computing the distance.
- Standardisation is more robust to outliers and in many cases, it is preferable over Max-Min Normalisation.
- In contrast to standardisation, we will obtain smaller standard deviations through the process of Max-Min Normalisation.
- Max-Min Normalisation typically allows us to transform the data with varying scales so that no specific dimension will dominate the statistics, and it does not require making a very strong assumption about the distribution of the data, such as k-nearest neighbours and artificial neural networks.
- However, Normalisation does not treat outliners very well.
- On the contrary, standardisation allows users to better handle the outliers and facilitate convergence for some computational algorithms like gradient descent.
- Therefore, we usually prefer standardisation over Min-Max Normalisation.
- The result of Standardization (or Z-score normalization) is that the features will be rescaled to ensure the mean and the standard deviation are 0 and 1, respectively.
- Data Standardization is a data processing workflow that converts the structure of disparate datasets into a Common Data Format.
- Standardization comes into picture when features of input data set have large differences between their ranges, or simply when they are measured in different measurement units (e.g., Pounds, Meters, Miles … etc).
- We try to bring all the variables or features to a similar scale. standarisation means centering the variable at zero.
-
- This technique to rescale features value with the distribution value between 0 and 1 is useful for the optimization algorithms, such as gradient descent, that are used within machine-learning algorithms that weight inputs (e.g., regression and neural networks).
- Rescaling is also used for algorithms that use distance measurements, for example, K-nearest-neighbours (KNN).
-
In Machine learning or Deep Learning, some of the models such as Linear Regression, Logistic Regression, Artificial Neural Networks assume that features are normally distributed and can perform much better if the features provided to them during modeling are normally distributed.
-
A normal distribution is sometimes informally called a bell curve.
-
Samples of the Gaussian Distribution follow a bell-shaped curve and lies around the mean. The mean, median, and mode of Gaussian Distribution are the same.
-
In probability theory, a normal (or Gaussian) distribution is a type of continuous probability distribution for a real-valued random variable.
- Log Transformation
- Square root Transformation
- Reciprocal Transformation
- Exponential Transformation
- Box-Cox Transformation
- Histogram
- Q-Q plot
- KDE plot
- Skewness and Kurtosis
- Five Number Summary
-
We call this Bell-shaped curve a Normal Distribution. Carl Friedrich Gauss discovered it so sometimes we also call it a Gaussian Distribution as well.
-
Bell Curve
-
-
The normal distribution is a core concept in statistics, the backbone of data science.