What are C and gamma with regards to a support vector machine?

https://www.quora.com/What-are-C-and-gamma-with-regards-to-a-support-vector-machine

 

 

C: regularization parameter. 즉, C가 크면 아주 strict하게 나누고, 작으면, 오판된 데이터를 잘 허용한다

gamma: window size for gaussian kernel. r = 1/(2*sigma^2). 따라서, sigma가 크면(gamma가 작으면) 윈도우 사이즈가 커져서 support vector의 영향력이 크게 된다.

What are C and gamma with regards to a support vector machine?

UpdateCancel

Answer Wiki

4 Answers

Jeffrey M GirardPsychology PhD Student

Studied at University of Pittsburgh

Lives in Pittsburgh, PA

291.5k answer views12k this month

Most Viewed Writer

Facial Expressions,

Support Vector Machines

Jeffrey M Girard, Affective Computing Researcher

Updated Apr 23, 2016

C and Gamma are the parameters for a nonlinear support vector machine (SVM) with a Gaussian radial basis function kernel.

A standard SVM seeks to find a margin that separates all positive and negative examples. However, this can lead to poorly fit models if any examples are mislabeled or extremely unusual. To account for this, in 1995, Cortes and Vapnik proposed the idea of a "soft margin" SVM that allows some examples to be "ignored" or placed on the wrong side of the margin; this innovation often leads to a better overall fit. C is the parameter for the soft margin cost function, which controls the influence of each individual support vector; this process involves trading error penalty for stability.

From: SVM - hard or soft margins?

A standard SVM is a type of linear classification using dot product. However, in 1992, Boser, Guyan, and Vapnik proposed a way to model more complicated relationships by replacing each dot product with a nonlinear kernel function (such as a Gaussian radial basis function or Polynomial kernel). Gamma is the free parameter of the Gaussian radial basis function.

A small gamma means a Gaussian with a large variance so the influence of x_j is more, i.e. if x_j is a support vector, a small gamma implies the class of this support vector will have influence on deciding the class of the vector x_i even if the distance between them is large. If gamma is large, then variance is small implying the support vector does not have wide-spread influence. Technically speaking, large gamma leads to high bias and low variance models, and vice-versa.