SVM
Support vector machine
SVM are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. The combination of the kernel trick plus a modified loss function, which is designed for binary classification.
A Support Vector Machine is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data supervised learning, the algorithm outputs an optimal hyperplane which categorizes new examples.
Support vector
Predictions only depend on a subset of the training data which is support vector. The vectors (cases) that define the hyperplane are the support vectors, which are cloest to the hyperplane.
Kernel Trick
The kernel function transform the data into a higher dimensional feature space to make it possible to perform the linear separation. It means a non-linear function is learned by a linear learning machine in a high-dimensional feature space while the capacity of the system is controlled by a parameter that does not depend on the dimensionality of the space.
####Hard Margin and Soft Margin
Hard margin SVM can work only when data is completely linearly separable without any errors (noise or outliers). In case of errors either the margin is smaller or hard margin SVM fails. On the other hand soft margin SVM solve this problem by introducing slack variables.
Train
Maximize the margin between the data points and the hyperplane. The loss function that helps maximize the margin is hinge loss. The objective of the regularization parameter is to balance the margin maximization and loss.
Some questions
- What is suppor vector?
- What is the idea of SVM?
- Explain the idea of kernel method?
- What is the slack factor in SVM?
- What is the loss function of SVM?