CSC522_HW2

ADLA – Spring 2024 Homework 2 Last Updated: February 15, 2024 2 1-NN, & Cross Validation (15 points) Consider the following dataset (9 instances) with 2 continuous attributes ( x 1 and x 2 ) that have been scaled to be in the same range, and a class attribute y , shown in Table 1. For this question, we will consider a 1-Nearest-Neighbor (1-NN) classifier that uses euclidean distance. Table 1: 1-NN ID x1 x2 Class 1 5.56 1.25 + 2 3.61 3.33 - 3 8.06 5 - 4 3.89 4.17 + 5 10 7.5 - 6 2.78 7.08 - 7 1.94 0 + 8 2.22 6.25 + 9 6.11 4.17 - 2a) Calculate the distance matrix for the dataset using euclidean distance. Tip : You can write a simple program to do this for you, and there is an example of how to do this in the programming portion of this homework. 2b) By hand, evaluate the 1-NN classifier, calculating the confusion matrix and testing accuracy (show your work by labeling each data object with the predicted class). Tip : you can scan a row or column of the distance matrix to easily find the closest neighbor. Use the following evaluation methods: i) A holdout test dataset consisting of last 4 instances ii) 3-fold cross-validation, using the following folds with IDs: [1,2,3], [4,5,6], [7,8,9] respectively. iii) Leave one out cross validation (LOOCV) 2c) For a data analysis homework, you are asked to perform an experiment with a binary classification algorithm that uses a “simple majority vote classifier“ which always predicts the majority class in the training dataset (if there is no majority, one of the classes is chosen at random). You are given a dataset with 50 instances and a class attribute that can be either Positive or Negative. The dataset includes 25 positive and 25 negative instances. You use three different validation methods: holdout (with a random 30/20 training/validation split), 5-fold cross validation (with random folds) and LOOCV. You expect the simple majority classifier to achieve approximately 50% validation accuracy, but for one of these evaluation methods you get 0% validation accuracy. Which evaluation gives this results and why? 3