Support Vector Machine - Mat Lab Code

Full SVM implementation

Today I'm going to present you how to implement a simple Support Vector Machine for 'XOR' Gate in basic. Actually there is a built-in command for SVM in Matlab. but you can't find a clear understanding about SVM through it.

% SVM - XOR
%
% kernel function :- k(x,y) = (1+x.y)^2
% X1  X2  Y    
% 0   0   1
% 0   1   -1
% 1   0   -1
% 1   1   1
%
% t1=1, t2=-1, t3=-1, t4=1 and x1=[0,0] , x2=[0,1] , x3=[1,0] , x4=[1,1]
%
% alpha = al
% al1,al2,al3,al4
%
%kfun=(1+x*y)^2;

clc;
t(1)=1;
t(2)=-1;
t(3)=-1;
t(4)=1;

X1 =[0;0];
X2 =[0;1];
X3 =[1;0];
X4 =[1;1];

C=cell(1,4);

C{1,1}=X1;
C{1,2}=X2;
C{1,3}=X3;
C{1,4}=X4;

Val1=cell(4,1);
Val2=cell(4,1);

%simplifying calculation to obtain kernel function

for num=1:4
    s1=C{1,num}(1,1)^2;
    s2=sqrt(2)*C{1,num}(1,1)*C{1,num}(2,1);
    s3=C{1,num}(2,1)^2;
    s4=sqrt(2)*C{1,num}(1,1);
    s5=sqrt(2)*C{1,num}(2,1);

   Val1{num,1}=[1;s1;s2;s3;s4;s5];
   Val2{num,1}=Val1{num,1};

end

%values of kernel function

for i=1:4
    for j=1:4
        multi=0;
        targe{i,j}= t(i)*t(j);
      for num=1:6
         multi=double( Val1{i,1}(num,1)*Val1{j,1}(num,1)+multi);
      end
      K{i,j}= multi;
    end
end

t=cell2mat(targe)
Kval=cell2mat(K)
Hessian= Kval.*t

% Making appropriate matices to perform quadprog
 f = [-1; -1; -1;-1];

 %constraints with equality
 Aeq=[1 -1 -1 1];
 Beq=[0];

 %lower bound for alphas
 lb = zeros(4,1);

% solving quadratic programming problem
 [x,fval,exitflag,output,lambda] = quadprog(Hessian,f,[],[],Aeq,Beq,lb);

% solutions for alphas
 al=x 
 count=0;

 for i=1:4
     if isempty(al(i))
     else
         count=count+1;
     end   
 end

 disp( sprintf( 'Number of support vector machines Ns= %d', count ) ); 

sum=0;

%Calculating Weight

for i=1:4
   sum=al(i)*t(i)*Val1{i,1}+sum ;
end 

disp( sprintf( 'Weight of support vector machine' ) ); 
Weight=sum


%Calculating Bias

calc=0;
fin=0;
for i=1:4
    for j=1:4
        calc=al(j)*t(j)*Kval(i,j)+calc;
    end
    fin=t(i)-calc+fin;
    calc=0;
end

disp( sprintf( 'Bias of support vector machine' ) ); 
bias=fin/4

%testing svm
 out1=transpose(Weight)*Val1{1,1}+bias;
 out2=transpose(Weight)*Val1{2,1}+bias;
 out3=transpose(Weight)*Val1{3,1}+bias;
 out4=transpose(Weight)*Val1{4,1}+bias;

%printing resutls

 disp( sprintf( 'Output resuts of svms') ); 
 disp( sprintf( 'Input X1= %d', out1 ) ); 
 disp( sprintf( 'Input X2= %d', out2 ) ); 
 disp( sprintf( 'Input X3= %d', out3 ) ); 
 disp( sprintf( 'Input X4= %d', out4 ) ); 
 

Download Project Files

 Project Files 

Quadratic Problem

Multiple SVM Classification (One Vs All)

 

3 Classes classification

for i=1:3
  svmStruct(i) = svmtrain(RBFoutput,group{i,1},'showplot',true);
end

for j=1:size(RBFoutput)
    for k=1:3
       if(svmclassify(svmStruct(k),RBFoutput(j,:)))
         break;
       end
    end
   result(j) = k;
end