SVM and Kernel Methods

In this assignment you will implement and experiment with Support Vector Machines (SVM) and kernel methods.

You will cover:

• Hard-margin SVM: Formulate the maximum-margin classification problem and understand the role of support vectors.
• Soft-margin SVM: Introduce slack variables and the regularization parameter \(C\) to handle non-separable data.
• Kernel trick: Apply the RBF, polynomial, and linear kernels to map data into higher-dimensional feature spaces without explicit transformation.
• Dual formulation: Understand the dual problem and how the kernel function appears in it.
• Evaluation: Compare SVM with different kernels on classification benchmarks and analyze decision boundaries.

Problems will be released closer to the due date.