$\ell_{0}$-Norm Penalty Embedded Feature Selection in Universum Learning

Universum learning exploits an auxiliary dataset, whose samples lie within the same domain as the target classes but are not labeled as either, thereby injecting valuable prior knowledge into the learning process. In high-dimensional settings, however, a significant challenge arises regarding whether the Universum has positive effects on effectively constraining the space of admissible models. To settle this problem, this paper introduces anℓ0-norm penalty into the Universum support vector machine, enabling the classifier to implement Universum in an essentially lower-dimensional space constituted by the most relevant variables. To derive practical solutions of the prosed NP-hard problem, we establish the existence of an optimal solution and subsequently formulate necessary and sufficient optimality conditions by introducing two types of stationary points. These optimality conditions facilitate a practical approach for solving the proposed model. Accordingly, we develop an effective alternating direction method of multipliers algorithm. The convergence of this algorithm is demonstrated under specific conditions in both theory and practice. Numerical experiments across various datasets demonstrate the effectiveness of our proposed method.