Hour 6 Support Vector Machine

####Concept

Support Vector Machines (SVM) are supervised learning models used for classification and regression tasks. The goal of SVM is to find the optimal hyperplane that maximally separates the classes in the feature space. The hyperplane is chosen to maximize the margin, which is the distance between the hyperplane and the nearest data points from each class, known as support vectors.

For nonlinear data, SVM uses a kernel trick to transform the input features into a higher-dimensional space where a linear separation is possible. Common kernels include:
- Linear Kernel
- Polynomial Kernel
- Radial Basis Function (RBF) Kernel
- Sigmoid Kernel

## Implementation Example

Suppose we have a dataset that records features like petal length and petal width to classify the species of iris flowers.

# Import necessary libraries

import numpy as np

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.svm import SVC

from sklearn.metrics import accuracy_score, confusion_matrix,

classification_report

import matplotlib.pyplot as plt

import seaborn as sns

# Example data (Iris dataset)

from sklearn.datasets import load_iris

iris = load_iris()

X = iris.data[:, 2:4]  # Using petal length and petal width as features

y = iris.target

# Splitting the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,

random_state=0)

# Creating and training the SVM model with RBF kernel

model = SVC(kernel='rbf', C=1.0, gamma='scale', random_state=0)

model.fit(X_train, y_train)

# Making predictions

y_pred = model.predict(X_test)

# Evaluating the model

accuracy = accuracy_score(y_test, y_pred)

conf_matrix = confusion_matrix(y_test, y_pred)

class_report = classification_report(y_test, y_pred)

print(f"Accuracy: {accuracy}")

print(f"Confusion Matrix:\n{conf_matrix}")

print(f"Classification Report:\n{class_report}")

# Plotting the decision boundary

def plot_decision_boundary(X, y, model):

    h = .02  # step size in the mesh

    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1

    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),

np.arange(y_min, y_max, h))

    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])

    Z = Z.reshape(xx.shape)

    plt.contourf(xx, yy, Z, alpha=0.8)

    sns.scatterplot(x=X[:, 0], y=X[:, 1], hue=y, palette='bright',

edgecolor='k', s=50)

    plt.xlabel('Petal Length')

    plt.ylabel('Petal Width')

    plt.title('SVM Decision Boundary')

    plt.show()

plot_decision_boundary(X_test, y_test, model)

Results

Accuracy: 1.0

Confusion Matrix:

[[11  0  0]

 [ 0 13  0]

 [ 0  0  6]]

Classification Report:

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        11

           1       1.00      1.00      1.00        13

           2       1.00      1.00      1.00         6

    accuracy                           1.00        30

   macro avg       1.00      1.00      1.00        30

weighted avg       1.00      1.00      1.00        30

Plot

#### Explanation of the Code

1. Importing Libraries

2. Data Preparation

3. Train-Test Split

4. Model Training: We create an SVC model with an RBF kernel (kernel='rbf'), regularization parameter C=1.0, and gamma parameter set to 'scale', and train it using the training data.

5. Predictions: We use the trained model to predict the species of iris flowers for the test set.

6. Evaluation: We evaluate the model using accuracy, confusion matrix, and classification report.

7. Visualization: Plot the decision boundary to visualize how the SVM separates the classes.

#### Decision Boundary

The decision boundary plot helps to visualize how the SVM model separates the different classes in the feature space. The SVM with an RBF kernel can capture more complex relationships than a linear classifier.

SVMs are powerful for high-dimensional spaces and effective when the number of dimensions is greater than the number of samples. However, they can be memory-intensive and require careful tuning of hyperparameters such as the regularization parameter \(C\) and kernel parameters.

Cracking the Data Science Interview 

👇👇

https://topmate.io/analyst/1024129

Credits: t.me/datasciencefun

ENJOY LEARNING 👍👍