ML_assignment_6.pdf

Machine Learning Assignment 6 - Probabilistic Models Fall 2023 Meet Sakariya 14473322 1 Theory 1 (a) Computer the posteriors for the observation i. Inference (5pts) Solution: Posterior using Inference: P(y=+ |x=[T,T]) = 3/12 = 0.25 P(y=- |x=[T,T]) = 0/12 = 0 Normalize: P(y=+ |x=[T,T]) = 0.25/(0.25+0) = 1 P(y=- |x=[T,T]) = 0/(0.25+0) = 0 ii. Naive Bayes (5pts) Solution: Posterior using Naive Bayes: P(y=+) = 12/21 = 0.571 P(x1=T y=+) = (3+4)/12 = 0.583 P(x2=T y=+) = (3+4)/12 = 0.583 x = [ ] using: 1. Consider the following set of training examples for an unknown target function: ( Y x1 x2 Count + T T 3 + T F 4 + F T 4 + F F 1 - T T 0 - T F 1 - F T 3 - F F 5 ) : T,T x1, x2 y | | →

2 P(y=-) = 9/21 = 0.428 P(x1=T |y=-) = 1/9 = 0.111 P(x2=T |y=-) = 3/9 = 0.333 Naive Bayes for x = [T, T] P(Y |X) = (P(Y).P(X|Y))/P(X) P(Y=+).P(x1=T|Y=+).P(x2=T|Y=+) and P(Y |X) = (P(Y).P(X|Y))/P(X) P(Y=-).P(x1=T|Y=-).P(x2=T|Y=-) P(y=+ |x=[T,T]) = (0.571)(0.583)(0.583) = 0.194 P(y=- |x=[T,T]) = (0.428)(0.111)(0.333) = 0.016 Normalize: P(y=+ x) = 0.194/(0.194+0.016) = 0.924 P(y=- x) = 0.016/(0.194+0.016) = 0.076 | |

4

3 AdditionalDataset 5 Now let’s see how your systems work on another multi-class dataset, the Yale Faces dataset! As with the prior assignment, read in all the images, resize them to be 40 × 40 images (use nearest neighbors so no interpolation occurs), then reshape into 1 × 1600 feature vectors, such that in the end we have a 154 × 1600 observable data matrix. As far as pre-processing the features, as long as you used nearest neighbors for resizing your images, all the features will be discrete [0,255], so there’s no need to zscore or convert features to categorical. We’ll again be classifying the subject, so again you should divide your training and validation ac- cording to subject (2/3 of each subject for training, 1/3 for validating). Plug this into your Naive Bayes implementation and again report the validation accuracy and confusion matrix. Solution: Here are the main pre-processing steps I performed on the Yale Faces dataset: 1) I extracted the subject ID from the filename of each image to get the class label 2) I resized each image to 40x40 pixels using nearest neighbor interpolation to keep the pixel values discrete while reducing the image size. 3) Each 40x40 image was flattened into a 1D array of length 1600 (40 * 40 pixels). 4) The dataset was split stratified by subject into 2/3 training and 1/3 validation sets to ensure the same distribution of subjects in each set. 5) Since the pixel values are discrete integers in [0,255], I did not standardize or normalize the feature Accuracy: 0.7843137254901961 Confusion matrix: