Scalable training of sparse linear SVMs

Guo Xun Yuan, Kwan-Liu Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Sparse linear support vector machines have been widely applied to variable selection in many applications. For large data, managing the cost of training a sparse model with good predication performance is an essential topic. In this work, we propose a scalable training algorithm for large-scale data with millions of examples and features. We develop a dual alternating direction method for solving L1-regularized linear SVMs. The learning procedure simply involves quadratic programming in the same form as the standard SVM dual, followed by a soft-thresholding operation. The proposed training algorithm possesses two favorable properties. First, it is a decomposable algorithm by which a large problem can be reduced to small ones. Second, the sparsity of intermediate solutions is maintained throughout the training process. It naturally promotes the solution sparsity by soft-thresholding. We demonstrate that, by experiments, our method outperforms state-of-the-art approaches on large-scale benchmark data sets. We also show that it is well suited for training large sparse models on a distributed system.

Original languageEnglish (US)
Title of host publicationProceedings - 12th IEEE International Conference on Data Mining, ICDM 2012
Pages775-784
Number of pages10
DOIs
StatePublished - Dec 1 2012
Event12th IEEE International Conference on Data Mining, ICDM 2012 - Brussels, Belgium
Duration: Dec 10 2012Dec 13 2012

Other

Other12th IEEE International Conference on Data Mining, ICDM 2012
CountryBelgium
CityBrussels
Period12/10/1212/13/12

Fingerprint

Quadratic programming
Support vector machines
Costs
Experiments

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Yuan, G. X., & Ma, K-L. (2012). Scalable training of sparse linear SVMs. In Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012 (pp. 775-784). [6413852] https://doi.org/10.1109/ICDM.2012.157

Scalable training of sparse linear SVMs. / Yuan, Guo Xun; Ma, Kwan-Liu.

Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012. 2012. p. 775-784 6413852.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yuan, GX & Ma, K-L 2012, Scalable training of sparse linear SVMs. in Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012., 6413852, pp. 775-784, 12th IEEE International Conference on Data Mining, ICDM 2012, Brussels, Belgium, 12/10/12. https://doi.org/10.1109/ICDM.2012.157
Yuan GX, Ma K-L. Scalable training of sparse linear SVMs. In Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012. 2012. p. 775-784. 6413852 https://doi.org/10.1109/ICDM.2012.157
Yuan, Guo Xun ; Ma, Kwan-Liu. / Scalable training of sparse linear SVMs. Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012. 2012. pp. 775-784
@inproceedings{0e672cd4b9ad46e7abd341fdcc886ce5,
title = "Scalable training of sparse linear SVMs",
abstract = "Sparse linear support vector machines have been widely applied to variable selection in many applications. For large data, managing the cost of training a sparse model with good predication performance is an essential topic. In this work, we propose a scalable training algorithm for large-scale data with millions of examples and features. We develop a dual alternating direction method for solving L1-regularized linear SVMs. The learning procedure simply involves quadratic programming in the same form as the standard SVM dual, followed by a soft-thresholding operation. The proposed training algorithm possesses two favorable properties. First, it is a decomposable algorithm by which a large problem can be reduced to small ones. Second, the sparsity of intermediate solutions is maintained throughout the training process. It naturally promotes the solution sparsity by soft-thresholding. We demonstrate that, by experiments, our method outperforms state-of-the-art approaches on large-scale benchmark data sets. We also show that it is well suited for training large sparse models on a distributed system.",
author = "Yuan, {Guo Xun} and Kwan-Liu Ma",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/ICDM.2012.157",
language = "English (US)",
isbn = "9780769549057",
pages = "775--784",
booktitle = "Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012",

}

TY - GEN

T1 - Scalable training of sparse linear SVMs

AU - Yuan, Guo Xun

AU - Ma, Kwan-Liu

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Sparse linear support vector machines have been widely applied to variable selection in many applications. For large data, managing the cost of training a sparse model with good predication performance is an essential topic. In this work, we propose a scalable training algorithm for large-scale data with millions of examples and features. We develop a dual alternating direction method for solving L1-regularized linear SVMs. The learning procedure simply involves quadratic programming in the same form as the standard SVM dual, followed by a soft-thresholding operation. The proposed training algorithm possesses two favorable properties. First, it is a decomposable algorithm by which a large problem can be reduced to small ones. Second, the sparsity of intermediate solutions is maintained throughout the training process. It naturally promotes the solution sparsity by soft-thresholding. We demonstrate that, by experiments, our method outperforms state-of-the-art approaches on large-scale benchmark data sets. We also show that it is well suited for training large sparse models on a distributed system.

AB - Sparse linear support vector machines have been widely applied to variable selection in many applications. For large data, managing the cost of training a sparse model with good predication performance is an essential topic. In this work, we propose a scalable training algorithm for large-scale data with millions of examples and features. We develop a dual alternating direction method for solving L1-regularized linear SVMs. The learning procedure simply involves quadratic programming in the same form as the standard SVM dual, followed by a soft-thresholding operation. The proposed training algorithm possesses two favorable properties. First, it is a decomposable algorithm by which a large problem can be reduced to small ones. Second, the sparsity of intermediate solutions is maintained throughout the training process. It naturally promotes the solution sparsity by soft-thresholding. We demonstrate that, by experiments, our method outperforms state-of-the-art approaches on large-scale benchmark data sets. We also show that it is well suited for training large sparse models on a distributed system.

UR - http://www.scopus.com/inward/record.url?scp=84874082603&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874082603&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2012.157

DO - 10.1109/ICDM.2012.157

M3 - Conference contribution

SN - 9780769549057

SP - 775

EP - 784

BT - Proceedings - 12th IEEE International Conference on Data Mining, ICDM 2012

ER -