ML_assignment_3.pdf

Machine Learning Assignment 3 - Logistic Regression and Gradient-Based Learning Summer 2023 Meet Sakariya - 14473322 1 1. For the function J = (x1w1 −5x2w−2)22,wherew=[w1,w2]areourweightstolearn: (a) What are the partial gradients, ∂J and ∂J ? Show work to support your answer (6pts). ∂w1 ∂w2 Solution: Given: J = (x1w1 − 5x2w2 − 2)2 First we will calculate partial gradient of J with respect to w1 ∂J = 2(x1w1 −5x2w2−2)(x1−0−0)∂w1 ∂J = 2x1(x1w1 −5x2w2−2)∂w1 ∂J =2x21w1 −10x1x2w2−4x∂w11 Similarly, we will calculate partial gradient of J with respect to w2 ∂J = 2(x1w1 −5x2w2−2)(0−5x2−0)∂w2 ∂J = −10x2(x1w1−5x2w2−2)∂w2 ∂J =50x22w2 −10x+20∂w1x2w1x22 (b) What are the values of the partial gradients, given current values of w = [0,0],x = [1,1] (4pts)? Solution: ∂J = 2 ∗ 12 ∗ 0−10 ∗ 1 ∗ 1 ∗ 0−4 ∗ 1∂w1 ∂J = −4∂w1 ∂J =50 ∗ 12 ∗ 0−10 ∗ 1 ∗ 1 ∗ 0+20 ∗ 1∂w2 ∂J =20 ∂w2 Values of partial gradients at w = [0,0],x = [1,1] are ∂J = −4and∂J=20∂w1∂w2 1 Theory

2 Logistic Regression Threshold: 50% In the code, I have taken: 2 1. Reads in the data. 2. Randomizes the data. 3. Selects the first 2/3 (round up) of the data for training and the remaining for validation. 4. Standardizes (z-scores) the data (except for the target column of course) using the training data. 5. Trains a logistic classifier, keeping track of the mean log loss for both the training and validation data as you train. 6. Classifies each validation sample using your trained model, choosing an observation to be spam if the output of the model is ≥ 50%. 7. Computes the following statistics using the validation data results: (a) Precision (b) Recall (c) F-measure (d) Accuracy (expect around 90%) 8. Plots epoch vs mean log-loss of both the training and validation data sets. 1. Seed the random number generate with zero prior to randomizing the data 2. We will let you determine appropriate values for the learning rate, η , the initial parameter values, as well as an appropriate termination criteria. 3. You’ll need to figure how to deal with log(0) issues. In your report you will need: The statistics requested for your Logistic Classifier. The plot of epoch vs log-loss for the training and validation data sets (on the same graph). Lets design, implement, train and test a Logistic Regression Classifier. For training and validation, we’ll use the dataset mentioned in the Dataset section, but your code should work on any dataset that lacks header information and has several comma-separated continuous-valued features followed by a class id 0,1. Solution: Write a script that: Implementation Details •