Support Vector Machines, K-Nearest Neighbors, and Naive Bayes' Classifier Algorithms

The basic terminology of the support vector machine is introduced, including hyperplane support vectors, hard and soft margins, hinge loss, slack variables, and kernel. Dot product of two vectors https://www.w3.org/1998/Math/MathML" display="inline"> X → and https://www.w3.org/1998/Math/MathML" display="inline"> Y → is defined as a projection of https://www.w3.org/1998/Math/MathML" display="inline"> X → on https://www.w3.org/1998/Math/MathML" display="inline"> Y → . A decision rule is defined for the classification of a point as positive or negative. The main steps of an SVM classifier are discussed. The K -nearest neighbor (KNN) algorithm is a nonparametric algorithm based on the idea of proximity. Choosing a suitable k value is crucial to the algorithm's performance. Chief distance metrics are Euclidean distance, Manhattan distance, Minkowski distance, and Chebyshev distance. Applications of the KNN algorithm are mentioned. To understand the Naive Bayes classifier, concepts of probabilities in Bayesian statistics are required. Prior, conditional, and posterior probabilities are demarcated. A categorization of Bayes classifiers is based on the type of data handled, namely, Gaussian Naive Bayes classifier, multinomial Bayes classifier, and Bernoulli Bayes classifier. The calculation of the posterior probability is performed using the likelihood and prior probabilities for a given feature vector and a class variable. Applications of the Naive Bayes algorithm are presented. A comparison is made between support vector machines, KNNs , and Naive Bayes classifier algorithms.