Robust Least Squares Twin Support Vector Machine With Adaptive Pinball Loss

Compared with the conventional twin support vector machine (TSVM), the least squares TWSVM (LSTSVM) possesses the advantages of faster training speed and better generalization performance. However, the square loss function is used in the optimization problems of LSTSVM, which makes it sensitive to noise contained in training samples and thus suffers from poor generalization ability. In this article, a novel adaptive pinball loss function is proposed, which possesses the merits of both pinball and smooth pinball loss functions. Under a certain condition, the convexity of the adaptive pinball loss function is guaranteed. Furthermore, to improve the anti-noise and generalization abilities of LSTSVM, a robust LSTSVM (RLS-TWSVM) is developed by replacing the square loss function in LSTSVM with the adaptive pinball loss function. The iteratively reweighted least squares (IRLSs) method is adopted to solve the optimization problems of the proposed RLS-TWSVM. The computational complexity and convergence analysis for the training procedure of the RLS-TWSVM are both provided. Moreover, it is proven that the proposed method is insensitive to noise distributed near the classification boundary. It is also shown that the proposed method can simultaneously minimize within-class scatter and misclassification error. In experiments, the proposed method outperforms ten related methods in anti-noise and generalization. Especially, compared with the second-best methods, the proposed method achieves a total average accuracy increase of 3.28% and 1.16% on the original, feature noise-polluted, and label noise-polluted benchmark datasets and MINIST, respectively. The MATLAB codes for the proposed method can be downloaded from https://github.com/hbuxing/APL-RLSTSVM.