广西师范大学学报（自然科学版） ›› 2013, Vol. 31 ›› Issue (3): 51-58.

• • 上一篇    下一篇

非凸共轭梯度p范数正则化SVM分类算法

左信, 黄海龙, 刘建伟   

  1. 中国石油大学(北京)自动化研究所,北京102249
  • 收稿日期:2013-04-20 出版日期:2013-09-20 发布日期:2018-11-26
  • 通讯作者: 左信(1964—),男,安徽池州人,中国石油大学(北京)教授。E-mail:zuox@cup.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(21006127);中国石油大学(北京)基础学科研究基金资助项目(JCXK-2011-07)

Classifier of p-norm Regularizing SVM with Nonconvex Conjugate Gradient Algorithm

ZUO Xin, HUANG Hai-long, LIU Jian-wei   

  1. Research Institute of Automation,China University of Petroleum,Beijing 102249,China
  • Received:2013-04-20 Online:2013-09-20 Published:2018-11-26

摘要： 经典的p范数支持向量机分类算法的正则化阶次p往往被选定为0、1或2。但是通过大量的实验可知,p取0、1或2的分类效果并不一定是最佳的。针对不同的数据使用不同的正则化阶次,可以改进分类算法的预测准确率。刘建伟等从目前迭代再权的思想出发讨论了p范数正则化支持向量机问题,但由于每次求解的均是原问题的近似问题,因而得到的解是近似解。从最优化角度出发,应用非凸共轭梯度算法求解0＜p＜1时的p范数正则化支持向量机问题,分别对3种不同的支持向量机问题进行了求解,并通过处理3种典型的癌症数据集展示了算法的良好分类效果。

关键词: p范数, 支持向量机, 共轭梯度法

Abstract: Classical classification algorithm of SVM via p norm regularization usually takes the regularization parameter p as 0,1 or 2.However,large amount of experiments show that these parameters can not always achieve the best classification results.It means finding out the appropriate parameter according to specific dataset may help promote the predictive rate.LIU Jian-wei has already discussed this problem.However,as it is based on the idea of reweighed iteration,it only gets the approximate solution of the original problem.The original problem was solved from the point of optimization when 0＜p＜1.Three different kinds of SVM have been discussed and the classification results are shown with the experiments on three gene datasets.

Key words: p-norm, support vector machine, conjugate gradient algorithm

中图分类号: 

  • TP18
[1] BOSER B E,GUYON I M,VAPNIK V N.A training algorithm for optimal margin classifiers[C]//Proceedings of the Fifth Annual Workshop on Computational Learning Theory.New York:ACM Press,1992:144-152.
[2] CORTES C,VAPNIK V.Support-vector networks[J].Machine Learning,1995,20:273-297.
[3] WESTON J,ELISSEEFF A,SCHOLKOPF B,et al.Use of the zero-norm with linear models and kernel methods[J].Journal of Machine Learning Research,2003,3:1439-1461.
[4] ZHU Ji,HASTIE T,ROSSET S,et al.1-Norm support vector machines[C]//Neural Information Processing Systems.Cambridge,USA:MIT Press,2004:16.
[5] LIU Zhen-qiu,JIANG Feng,TIAN Guo-liang,et al.Sparse logistic regression with Lp penalty for biomarker identification[J].Statistical Applications in Genetics and molecular Biology,2007,6(1):1-22.
[6] NG A Y.Feature selection,L1 vs.L2 regularization,and rotational invariance[C]//Proc of 21st International Conference on Machine Learning.New York:ACM Press,2004:78.
[7] LIU Yu-feng,WU Yi-chao.Variable selection via a combination of the L0 and L1 penalties[J].Journal of Computational and Graphical Statistics,2007,16(4):782-798.
[8] LIU Yu-feng,ZHANG He-len,CHEOLWOO P,et al.Support vector machines with adaptive Lq penalties[J].Computational Statistics and Data Analysis,2007,51(12):6380-6394.
[9] FLETCHER R,REEVES C M.Function minimization by convergent gradients[J].The Computer Journal,1964,7(2):149-154.
[10] WOLFE P.Convergence conditions for ascent methods[J].SIAM Review,1969,11(2):226-235.
[11] WOLFE P.Convergence conditions for ascent methods Ⅱ:some corrections[J].SIAM Review,1969,13(2):185-188.
[12] MORE J J,THUENTE D J.Line search algorithms with guaranteed sufficient decrease[J].ACM Transactions on Mathematical Software,1994,20:286-307.
[13] HAGER W W,ZHANG Hong-chao.A new conjugate gradient method with guaranted descent and an efficient line search[J].SIAM Optimization,2005,16:170-192.
[14] PYTLAK R.Conjugate gradient algorithm in nonconvex optimization[M].Berlin:Springer,2009.
[15] 刘建伟,李双成,罗雄麟.p范数正则化支持向量机分类算法[J].自动化学报,2012,38:76-87.
[1] 朱勇建, 彭柯, 漆广文, 夏海英, 宋树祥. 基于机器视觉的太阳能网版缺陷检测[J]. 广西师范大学学报（自然科学版）, 2019, 37(2): 105-112.
[2] 吕凯晨, 闫宏飞, 陈翀. 基于沪深300成分股的量化投资策略研究[J]. 广西师范大学学报（自然科学版）, 2019, 37(1): 1-12.
[3] 李子彦, 刘伟铭. 一种基于局部HOG特征的运动车辆检测方法[J]. 广西师范大学学报（自然科学版）, 2017, 35(3): 1-13.
[4] 刘艳红, 罗晓曙, 陈锦, 郭磊. 宫颈细胞图像的特征提取与识别研究[J]. 广西师范大学学报（自然科学版）, 2016, 34(2): 61-66.
[5] 陈思溢, 罗强, 黄辉先. 基于群决策理论的协调控制子区划分方法[J]. 广西师范大学学报（自然科学版）, 2014, 32(4): 18-25.
[6] 王世明, 徐建闽, 李日涵. 城市快速路入口匝道控制算法的改进[J]. 广西师范大学学报（自然科学版）, 2012, 30(2): 1-6.
[7] 严晓明, 郑之. 基于混合仿生算法的SVM参数优化[J]. 广西师范大学学报（自然科学版）, 2011, 29(2): 114-118.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发