Lesson3.md

Deep Learning

Lesson 3

What is a Cost function?

Cost Function ia a method of evaluating how well a machine learning algorithm predicts the output by comparing the predicted target lables with the known target labels. If predictions are pretty good, loss function will have a lower value. If not,the loss function will have a high value.

• Multi-class SVM loss Multi-class SVM Loss is a cost or a loss function that uses a scoring function 'f' to determine the numerical scores for each target class labels.

For the cat prediction ,loss would be calculated as shown in the figure below

Regularization

Regularization is a process that is used to minimize the loss function and prevent overfitting or underfitting.

Types of Regularization

• L1 Regularization In L1 regularization(also known as Lassso Regression), a regularization term equal to the absolute value of magnitude of coefficient is added to the cost function as penalty term. L1 regularization can be used for feature selection.

• L2 Regularization In L2 regularization(also known as Ridge Regression), a regularization term equal to the square value of magnitude of coefficient is added to the cost function as penalty term. If we have features which are highly correlated top each other, we can use the L2 regularization as it allows the less signinficant features to have some influence at predicting the outcome rather than eliminating it all together.

Elastic Net Regularization Elastic Net Regularization is the combination of Lasso and Ridge regression.