Suppose that you have designed a new machine learning model for a binary classification task and would like to evaluate the error rate of this classifier. a. If your proposed model has an error rate of 9.7% on a balanced test dataset with 100 samples in total, construct a 95% confidence interval for the error rate of your classifier. b. Suppose that a state-of-the-art classifier has a reported error rate of 12% for this task. How many samples from the same test dataset should be correctly classified by your model before you can claim that your proposed model has a statistically better error rate than the state-of-the-art at a significance level of 0.05? c. Find the power of the above test if the true error rate of your model is 9%.
Suppose that you have designed a new machine learning model for a binary classification task and would like to evaluate the error rate of this classifier. a. If your proposed model has an error rate of 9.7% on a balanced test dataset with 100 samples in total, construct a 95% confidence interval for the error rate of your classifier. b. Suppose that a state-of-the-art classifier has a reported error rate of 12% for this task. How many samples from the same test dataset should be correctly classified by your model before you can claim that your proposed model has a statistically better error rate than the state-of-the-art at a significance level of 0.05? c. Find the power of the above test if the true error rate of your model is 9%.
MATLAB: An Introduction with Applications
6th Edition
ISBN:9781119256830
Author:Amos Gilat
Publisher:Amos Gilat
Chapter1: Starting With Matlab
Section: Chapter Questions
Problem 1P
Related questions
Question
Suppose that you have designed a new machine learning model for a binary classification task and would like to evaluate the error rate of this classifier.
a. If your proposed model has an error rate of 9.7% on a balanced test dataset with 100 samples in total, construct a 95% confidence interval for the error rate of your classifier.
b. Suppose that a state-of-the-art classifier has a reported error rate of 12% for this task. How many samples from the same test dataset should be correctly classified by your model before you can claim that your proposed model has a statistically better error rate than the state-of-the-art at a significance level of 0.05?
c. Find the power of the above test if the true error rate of your model is 9%.
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step 1: Write the given information.
VIEWStep 2: Determine the 95% confidence interval for the error rate of your classifier.
VIEWStep 3: Determine required sample size that has statistically better error rate than the state-of-the-art.
VIEWStep 4: Determine the power of the above test if the true error rate of your model is 9%.
VIEWSolution
VIEW
Step by step
Solved in 5 steps with 4 images
Recommended textbooks for you
MATLAB: An Introduction with Applications
Statistics
ISBN:
9781119256830
Author:
Amos Gilat
Publisher:
John Wiley & Sons Inc
Probability and Statistics for Engineering and th…
Statistics
ISBN:
9781305251809
Author:
Jay L. Devore
Publisher:
Cengage Learning
Statistics for The Behavioral Sciences (MindTap C…
Statistics
ISBN:
9781305504912
Author:
Frederick J Gravetter, Larry B. Wallnau
Publisher:
Cengage Learning
MATLAB: An Introduction with Applications
Statistics
ISBN:
9781119256830
Author:
Amos Gilat
Publisher:
John Wiley & Sons Inc
Probability and Statistics for Engineering and th…
Statistics
ISBN:
9781305251809
Author:
Jay L. Devore
Publisher:
Cengage Learning
Statistics for The Behavioral Sciences (MindTap C…
Statistics
ISBN:
9781305504912
Author:
Frederick J Gravetter, Larry B. Wallnau
Publisher:
Cengage Learning
Elementary Statistics: Picturing the World (7th E…
Statistics
ISBN:
9780134683416
Author:
Ron Larson, Betsy Farber
Publisher:
PEARSON
The Basic Practice of Statistics
Statistics
ISBN:
9781319042578
Author:
David S. Moore, William I. Notz, Michael A. Fligner
Publisher:
W. H. Freeman
Introduction to the Practice of Statistics
Statistics
ISBN:
9781319013387
Author:
David S. Moore, George P. McCabe, Bruce A. Craig
Publisher:
W. H. Freeman