Obtain Decision Boundary for SVM

svm decision boundary
svm parameters
how to find decision boundary in svm python
how to calculate weight vector in svm
decision boundary sklearn svm
sklearn svm get decision boundary
cubic svm
svm explained

In the following example: http://scikit-learn.org/stable/auto_examples/svm/plot_separating_hyperplane.html

I would like to obtain the coefficients of the (line) decision boundary shown in the picture. A call to

clf.coef_ 

returns

[[-0.2539717  -0.83806387]]

which, if I am not mistaken, represents the line of equation

y = -0.83806387 * x - 0.2539717

However, the above line is not the decision boundary obtained in the example, so what exactly is coef_ and how can I obtain the equation of the linear decision boundary ?

To get the equation for the line of the decision boundary of a linear model you need to get both the coef_ and intercept_. Also note that since you are using a SVC there will be multiple decision boundaries involved.

The line equation can be constructed as:

y = w0 + w1 * x1 + w2 * x2 + ...

Where w0 is obtained from intercept_, w1 onwards are found in coef_, and x1 and onwards are your features.

For example, this code shows you how to print out the equations for each of your decision boundaries.

from sklearn import svm
import numpy as np

clf = svm.SVC(kernel="linear")

X = np.array([[1, 2], [3, 4], [5, 1], [6, 2]])
y = np.array(["A", "B", "A", "C"])

clf.fit(X, y)

for (intercept, coef) in zip(clf.intercept_, clf.coef_):
    s = "y = {0:.3f}".format(intercept)
    for (i, c) in enumerate(coef):
        s += " + {0:.3f} * x{1}".format(c, i)

    print(s)

In this example, the lines are determined to be:

y = 2.800 + -0.200 * x0 + -0.800 * x1
y = 7.000 + -1.000 * x0 + -1.000 * x1
y = 1.154 + -0.462 * x0 + 0.308 * x1

Source: http://scikit-learn.org/stable/modules/linear_model.html

How to calculate decision boundary from support vectors?, I want to obtain decision boundary of SVM using OpenCV 2.4.11, but it seems that it's not returning it explicitly, but only support vectors. How we can calculate  To get the equation for the line of the decision boundary of a linear model you need to get both the coef_and intercept_. Also note that since you are using a SVC there will be multiple decision boundaries involved. The line equation can be constructed as: y = w0 + w1 * x1 + w2 * x2 +

If you want to draw linear plot i.e. y = ax + b then you can use below chunk of code

tmp = clf.coef_[0]
a = - tmp[0] / tmp[1]

b = - (clf.intercept_[0]) / tmp[1]

xx = np.linspace(xlim[0], xlim[1])
yy = a * xx + b
plt.plot(xx, yy)

Hope this helps!

Support vector machines: The linearly separable case, Intuitively, a decision boundary drawn in the middle of the void between data By construction, an SVM classifier insists on a large margin around the decision boundary. To get a sense of how to do that, let us look at the actual geometry. This is exactly how SVM works, it draws a decision boundary which is a hyperplane between any two classes in order to separate them or classify them. this will serve as a good starting point

Thanks @Prem and @Christopher Wells - this was realy helpful. I combined both answers because in Prem's code is no y_target included (--> see Christopher's answer), which works fine for y_target=0.

As an example I used the logRegression tutorial from scikitLearn. I inserted the following code:

mf = logreg.intercept_.shape[0];
xm = np.r_[np.min(X),np.max(X)]
yf = logreg.classes_.copy()
xm = np.r_[np.min(X),np.max(X)]

for jf in np.arange(mf):
    tmp = logreg.coef_[jf]
    a = - tmp[0] / tmp[1]; 
    b = - (logreg.intercept_[jf]-yf[jf]) / tmp[1]
    yy = a * xm + b
    plt.plot(xm, yy, label='Coeff ='+str(jf))
plt.legend()
plt.show()

This is perfect for y_target=0 (see graphic in the skLearn example). But what is with the other 2 straight lines ? Has anything more to be considerd ?

Support Vector Machines(SVM), It controls the trade off between smooth decision boundary and classifying training points correctly. A large value of c means you will get more  Our labeled data. A support vector machine takes these data points and outputs the hyperplane (which in two dimensions it’s simply a line) that best separates the tags. This line is the decision boundary: anything that falls to one side of it we will classify as blue, and anything that falls to the other as red.

Finding Non-Linear Decision Boundary in SVM, to get to Non-Linear decision boundaries, now that's absolutely ok, the only issue is how do we choose which features to add means which  Support Vector Machine (SVM) Support vectors Maximize margin •SVMs maximize the margin (Winston terminology: the ‘street’) around the separating hyperplane. •The decision function is fully specified by a (usually very small) subset of training samples, the support vectors. •This becomes a Quadratic programming problem that is easy

SVM: Maximum margin separating hyperplane, Paired) # plot the decision function ax = plt.gca() xlim = ax.get_xlim() ylim T Z = clf.decision_function(xy).reshape(XX.shape) # plot decision boundary and  clf = svm.SVC (kernel='linear', C=1) If you set C to be a low value (say 1), the SVM classifier will choose a large margin decision boundary at the expense of larger number of misclassifications. When C is set to a high value (say 100), the classifier will choose a low margin decision boundary and try to minimize the misclassifications.

Decision boundary, In a statistical-classification problem with two classes, a decision boundary or decision surface while support vector machines try to learn the decision boundary which maximizes the empirical margin between the decision boundary and  Those new features are the key for SVM to find the nonlinear decision boundary. In Sklearn — svm.SVC(), we can choose ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable as our kernel/transformation.

Comments
  • Tx for your explanation ! Yes, that helps. For a test with an example see my answer..
  • Yes, that helps. For a test with an example see my answer.