ECEN289 F22 HW3

docx

School

Texas A&M University *

*We aren’t endorsed by this school

Course

289

Subject

Mechanical Engineering

Date

Apr 3, 2024

Type

docx

Pages

Uploaded by AgentStorkPerson359

Max Kotas 530000126 ECEN 289: Machine Learning for EEs Homework 3 Assigned 9/29/22 Due 10/6/22 Potential Points 50 Description Description : classification errors, model selection, determining parameters, and naïve Bayes inferencing. Complete all homework questions in the space provided and upload a pdf of the completed docu- ment on canvas by 11:59pm on the due date. 1) classification performance metrics: You developed a classification model which uses images of damaged vehicles at accident sites to estimate the location of the initial impact to the vehicle. The images are combined with informa- tion from the vehicle – speed and direction of the vehicle prior to the initial impact to classify each damaged vehicle by location of initial impact. This classification will be used in determining the cause of the accident. The classes are {head-on, rear-end, left-front, right-front, right-rear, and left-rear. Your model was validated with video footage from intersections traffic cameras by crash engineers. 817 vehicle crash cases were classified by your model. The following are the results from your validated classification model: True Class Your model estimated Head-on = 154 Head-on = 130 Left- front=13 Right-front 11 Rear-end = 118 Rear- end=97 Right- rear=11 Left-rear=9 Left- front=1 Left- front=96 Left- front=86 Head-on=9 Right- front=1 Right- front=87 Right- front=59 Head- on=20 Left- front=8 Right- rear=143 Right- rear=121 Right- front=14 Rear-end=8 Left- rear=219 Left- rear=200 Rear- end=13 Right- rear=6

Max Kotas 530000126 (a) (5 points) Create a confusion matrix for your classifier results: [ 130 0 13 11 0 0 0 97 1 0 11 9 9 0 86 1 0 0 20 0 8 59 0 0 0 8 0 14 121 0 0 13 0 0 6 200 ] (b) (5 points) Compute the mean accuracy of your classification (show your work): accuracy for row = true positives/(true positives + false positives) mean accuracy = sum(accuracy for a row for each row)/n rows mean accuracy of model = 0.8273012163 2) understanding overfitting. You have developed 6 models for classifying the causes of electrical insulator failures in power transformers. Assume that each of the causes of failure are approximately equally likely (i.e. balanced dataset) in this case. You have trained these models on 3756 samples and validated the results on 1246 samples that have been analyzed by experts on insulator failure. Your mod- els give the following results: Model Training Accu- racy Validation accu- racy Training-F1 Validation-F1 1 0.83 0.76 0.81 0.73 2 0.94 0.72 0.9 0.71 3 0.81 0.74 0.8 0.74 4 0.79 0.71 0.79 0.7 5 0.96 0.76 0.92 0.74 6 0.80 0.75 0.78 0.72 (a) (3 pts) Which model will you present to your boss as the best model for predicting the cause of insulator failure, and why: I would use model 5, because the difference between the validation accuracy and the validation F1 is 0 with respect to the difference in accuracies

Max Kotas 530000126 (b) (3 pts) If you found out that the causes were not equally probable (i.e. imbalanced dataset), would you make a different choice? Justify your answer: If I found out that the causes of failure were not equally probable then I would be more likely to pick one with a higher validation accuracy because the dataset would be much more diverse and if it can extrapolate the results from its training to a higher accuracy in the real world then it ob- jectively a better model 3) You are building a vision system for a robot driven vehicle. One part of the system must recognize traffic signs. As a first demonstration of your system you will restrict your design to recognizing a subset of signs {stop, speed 25, speed 35, pedestrian crossing, merge ahead, yield, do not enter, no parking, Austin city limits, and railroad crossing}. You have driven a vehicle around the College Station area and you have counted the number of captured images of signs. These are the results: Sign Number Stop 400 Speed 25 300 Speed 35 300 Pedestrian crossing 300 Merge ahead 100 Yield 300 Do not enter 100 No parking 100 Austin city limits 0 Railroad crossing 100 (a) (5 points) Compute and present the table of the Prior probability distribution for the sign class Y for your observations. Sign Probability Stop 0.2 Speed 25 0.15 Speed 35 0.15 Pedestrian crossing 0.15

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Max Kotas 530000126 Merge ahead 0.05 Yield 0.15 Do not enter 0.05 No parking 0.05 Austin city limits 0 Railroad crossing 0.05 You have determine that you will combine the output of three sensors to help determine if an object is a sign and the type of the sign. This is, of course, a classification problem. The first sensor uses a camera and a computer vision library to detect shapes. It can detect {octagon, rectangle, diamond, circle, point-down triangle}, and others that you will not need for the demonstration. This sensor will output this value as F1=Shape. A second sensor is a color sensor from which you have selected the following colors to be output {red, green, white, yellow, orange}. If more than one color is detected it will output one or the other with equal probability. A third sensor is a camera with character recognition software which if it detects text will output the color of the text or the word none if it does not detect any text within the sign {white, black, none, red, green} as F3=Txt-color. Your analysis of the images shows that signs have these characteristics: Sign Shape Background color(s) Text color Stop Octagon Red White Speed 25 Rectangle White Black Speed 35 Rectangle White Black Pedestrian crossing Diamond Yellow None Merge ahead Diamond Yellow None Yield Point down triangle Red and white Red Do not enter Circle Red and white White No parking Rectangle White Red Austin city limits Rectangle Green White

Max Kotas 530000126 Railroad crossing Circle Orange Black You have developed and trained a Naïve Bayes classifier that will predict the class of a road sign based on the three features above. In computing the Probability distributions for the Features above you assume that the sensors will not make any errors in detection (probably a bad assumption!). (b) (5 points) Compute the Marginal distributions for feature F1 and present it as a probability distribution table. Octagon 0.1 Rectangle 0.4 Diamond 0.2 Down Trian- gle 0.1 Circle 0.2 (c) (5 points) Compute the Marginal distribution for feature F2 and present as a probability distribution table. Red 0.1 White 0.3 Yellow 0.2 Red and White 0.2 Green 0.1 Orange 0.1 (d) (5 points) Compute the Marginal distribution for feature F3 and present as a probability distribution table. White 0.3 Black 0.3 Red 0.2 None 0.2 (e) (5 points) You begin inferencing with your system – which begins with the Probabilities and Log- Probabilities for our Inferencing table from the Priors (As we did on slide 10 of lecture 10). The first evidence observed is from the Background color detector which indicates that a red background was detected. For each Y label {stop, speed 25, speed 35, pedestrian crossing, merge ahead, yield, do not enter, no parking, Austin city limits, and railroad crossing} compute the probability that the sign detected was of class Y and present in a probability distribution table: Sign P(Y|F1=Red) Stop 0.66 Speed 25 0 Speed 35 0

Max Kotas 530000126 Pedestrian crossing 0 Merge ahead 0 Yield 0.495 Do not enter 0.165 No parking 0 Austin city limits 0 Railroad crossing 0 (f) (4 points) A second piece of evidence arrives. The shape detector indicates that the shape of the detected object was a circle. With this new evidence, for each Y label {stop, speed 25, speed 35, pedestrian crossing, merge ahead, yield, do not enter, no parking, Austin city limits, and railroad crossing}, compute the probability that the sign detected was of class Y and present in a probability distribution table: Sign P(Y|F1=red, F2=circle) Stop 0 Speed 25 0 Speed 35 0 Pedestrian crossing 0 Merge ahead 0 Yield 0.75 Do not enter 0 No parking 0 Austin city limits 0 Railroad crossing 0 (g) (5 points) You decide that over fall break you are going to take your robot-driven vehicle to Big Bend National Park. About 90 miles into your 541 mile journey, the Background color detector indicates that it has detected a sign with a green background…..and the sign detection system crashes, the vision system crashes, and the robot driven safely pulls into a hotel parking lot. From your knowledge of how the sign detection system works, and since you set the probabilities that the system uses, explain the likely cause of the system failure and how you might have avoided this problem. The model trained so that the probability of seeing a green sign would be 0, and if we use naïve bayes theorem then then we would have to divide the equation by the probability of x which is the

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Max Kotas 530000126 sign color which = 0. It is impossible to divide by zero, but the programmer who wrote the code really should have accounted for that error. Once you have completed your analysis and entered it on this form, don’t forget to turn this into a pdf file and upload it to canvas by 11:59PM on 10/06