Perceptron Learning using the Delta rule - Gradient Descent - Java (Training for the boolean AND function)

The delta rule uses gradient descent to approximate the target function. It is guaranteed to converge even if the function is not linearly separable, unlike the perceptron learning rule.

We'll be training the perceptron to learn the boolean AND function. We'll use two weights w1, w2. These weights are approximated by the delta rule to fit the training examples most closely. Next we need to find the threshold value (w0) which we will use to classify the training examples accurately.

I've represented 0 with -1, 1 with 1 itself.

Code:

class Example{
    int x1, x2;
    double o;
    Example(int x1, int x2, double o){
        this.x1 = x1;
        this.x2 = x2;
        this.o = o;
    }
}

public class DeltaRulePerceptron {
    double w1, w2;
    double dw1, dw2;
    double n;
    
    DeltaRulePerceptron(){
        //Random values
        w1 = 0.1;
        w2 = 0.7;
        n = 0.05;
    }
    
    public double computeOutput(Example example){
        return example.x1 * w1 + example.x2*w2;
    }
    
    public void trainWithDelta(Example[] examples){ 
        for(int i=0;i<1000;++i){
           
//            System.out.println("Iteration #"+i);
            dw1 = 0;
            dw2 = 0;
            
            for(Example ex:examples){
                double o = computeOutput(ex);
                double t = ex.o;
//                System.out.println("o = "+o+" t = "+t); 
                
                dw1 = dw1 + n*(t-o)*ex.x1;
                dw2 = dw2 + n*(t-o)*ex.x2;
            } 
             
            w1 += dw1;
            w2 += dw2; 
        }        
    }
    
    public static void main(String[] args){
        DeltaRulePerceptron d = new DeltaRulePerceptron(); 
        
        //AND boolean function
        Example[] examples = new Example[]{
            new Example(-1, -1, -1),
            new Example(-1 , 1, -1),
            new Example( 1, -1, -1),
            new Example( 1,  1, 1)
        };
        d.trainWithDelta(examples);
        System.out.println("(AND) Trained weights : "+d.w1+" "+d.w2); 
        
        //bias
        double w0 = 0.5;
        System.out.println("w0 = "+w0);
        
        //Test
        System.out.println(d.sign(w0, d.computeOutput(examples[0])));
        System.out.println(d.sign(w0, d.computeOutput(examples[1])));
        System.out.println(d.sign(w0, d.computeOutput(examples[2])));
        System.out.println(d.sign(w0, d.computeOutput(examples[3])));
        
        
        //XOR - fails - A single layer perceptron can't represent a XOR function
        examples = new Example[]{
            new Example(-1, -1, 1),
            new Example(-1 , 1, -1),
            new Example( 1, -1, -1),
            new Example( 1,  1, 1)
        };
        d.trainWithDelta(examples);
        System.out.println("(XOR) Trained weights : "+d.w1+" "+d.w2);
        System.out.println("w0 = "+w0);
        
        System.out.println(d.sign(w0, d.computeOutput(examples[0])));
        System.out.println(d.sign(w0, d.computeOutput(examples[1])));
        System.out.println(d.sign(w0, d.computeOutput(examples[2])));
        System.out.println(d.sign(w0, d.computeOutput(examples[3])));
    }
    
    public int sign(double w0, double output){
        return output-w0>0?+1:-1;
    }
}

Output:

(AND) Trained weights : 0.49999999999999994 0.5000000000000002
w0 = 0.5
-1
-1
-1
1

(XOR) Trained weights : 0.0 5.551115123125783E-17
w0 = 0.5
-1
-1
-1
-1