After training a classifier model we have produced following confusion matrix where Y = 1 corresponds to a 'positive' prediction. Predicted Y = 0 Predicted Y = 1 Actual Y = 0 60 Actual Y = 1 60 75 (a) Determine the Accuracy, Precision Recall and F1 Score of your model (it's ok to leave the solution in fractional form like 1/10) (b) Analyze your model's performance. In what ways does it perform well and in what ways does it under-perform? (c) Propose a specific way you could try and improve the model's performance, and discuss how that approach might impact the other metrics.

After training a classifier model we have produced following confusion matrix where Y = 1 corresponds to a 'positive' prediction. Predicted Y = 0 Predicted Y = 1 Actual Y = 0 60 Actual Y = 1 60 75 (a) Determine the Accuracy, Precision Recall and F1 Score of your model (it's ok to leave the solution in fractional form like 1/10) (b) Analyze your model's performance. In what ways does it perform well and in what ways does it under-perform? (c) Propose a specific way you could try and improve the model's performance, and discuss how that approach might impact the other metrics.

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Similar questions

The following is true about sensitivity: Group of answer choices a) The output of the model is said to be inversely sensitive if the output of the model changes a small amount for a large change in an input variable b) Sensitivity is not an important concept in modeling c) It can help the modeler tell, on a relative basis, what are the important variables d) A variable is considered NOT very sensitive if a small change in the variable results `in a large change in the output of the model.
A model is used to generate a forecast using features as inputs and returning a prediction in a subset of machine learning called as is.... The following are some examples of new cutting-edge models that have become viable:
Question 48. Let us return to the Titanic data set. We now have learned several models and want to choose the best one. We used three different methods to validate these models: The training error rate (apparent error rate), the error rate on an external test set and the error rate estimated by a 10-fold cross validation. Training Error | Error on the test set | Cross Validation Error 0.18 Learner Decision Tree 0.22 0.21 Random Forest 0.01 0.10 0.12 1-Nearest-Neighbour 0.18 0.19 Which of the following statements are correct? a) 1-Nearest-Neighbour has a perfect training error and hence it should be used here. b) Random Forests outperforms both 1-Nearest-Neighbour and the Decision Tree in terms of prediction error. c) Not just in this case, but in general, Cross Validation is the better validation strategy and should always be preferred over the error on a single test set. d) Not just in this case, but in general, Decision Trees always perform worse than Random Forests.
Use R to solve the following tasks. TASK 1. Estimate the following probabilities by simulation (we don't want to see any analytical approaches!) In your report explain your code and show your answers. a. You have all the black cards from a normal deck, I have all the red cards. We each choose one card from our half decks - the highest card wins. But I have managed to sneak out and throw away your 9 Of Clubs! Simulate 1000 games - how many did I win? (Ace is low.) b. There are 87 people on a bus - what is the likelihood that two of them share a birthday? c. How many people would you need to gather together before there is a 90% chance that one of them was born on the 1st Jan?
None
The predictive performance of a model is the measure of how close the model’s prediction values are to the actual values. A close-to-ideal model would have the minimum error in the predicted and actual values. The validation set is used to assess the predictive ability of the model which has been trained using the training set. True False
You are developing a simulation model of a service system and are trying to create an input model of the customer arrival Process, You have the following four observations of the process of interest [86, 24,9, 50] and you are considering either an exponential distributionor a uniform distribution for the model. Using the data to estimate any necessary distribution Parameters, write the steps to plot Q-Q plots for both uniform and exponential distribution. Write the steps clearly. Thanks.
Use decision tree to further explore the dataset, where the dependent variable is ‘smoker’. Please explain the approach taken. [No more than 300 words]
In this section we create the model object and define the loss function and optimizer. For the loss function use nn.CrossEntropyLoss() instead of the mean-squared loss done in class. Use the same optimizer as in class. learning_rate = 0.001 num_epochs = 5 device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") #this is to use gpu if available # **create your neural net model here, call the variable 'model'** # ** complete following lines of code to define loss function and optimizer #criterion = **complete this** #optimizer = **complete this**
A team averaging 106 points is likely to do very well during the regular season. The coach of your team has hypothesized that your team scored at an average of less than 106 points in the years 2013-2015. Test this claim at a 1% level of significance. For this test, assume that the population standard deviation for relative skill level is unknown. Use the information in the picture to write a block of code. Here is some information that will help you write this code block. Reach out to your instructor if you need help. The dataframe for your team is called your_team_df. The variable 'pts' represents the points scored by your team. Calculate and print the mean points scored by your team during the years you picked. Identify the mean score under the null hypothesis. You only have to identify this value and do not have to print it. (Hint: this is given in the problem statement) Assuming that the population standard deviation is unknown, use Python methods to carry out the hypothesis…
Machine Learning Evaluation measures. Consider a dataset with 90 negative examples and 10 positive examples. Suppose a model built using this data predicts 30 of the examples as positive (only 10 of them are actually positive) and 70 as negative. What are the numbers of True Positives (TP), False Positives (FP), True Negatives (TN), False Negatives (FN), Accuracy, Precision, Recall, and Specificity? Show all your steps
Match the metrics with their corresponding definition. TPR/Recall PPV/Precision TNR/Specificity Accuracy [Choose ] The ratio of correctly classified predictions over all predictions The ratio of the positive predictions that were correct over the total number of positive predictions made. The ratio over the number of negative predictions that were correct over all the true negative class samples. The ratio over the number of positive predictions that were correct over all the true positive class samples [Choose ]