Answer the given question with a proper explanation and step-by-step solution.
Answer the given question with a proper explanation and step-by-step solution.
Now that you have written functions for different steps of the model building process you will put it all together. You will write code that trains a model with hyperparameters you determine (you should do any tuning locally or in a notebook ie don't tune your model in gradescope since the autograder will likely timeout). It will take in the CLAMP training data, train a model then predict on a test set and output values from 0 to 1 for each row and our autograder will compare your predictions with the correct answers and to get credit you will need a roc auc score of .9 or higher on the test set (should not require much hyperparameter tuning for this dataset). This is basically a simulation of how your model would perform in the “production” system using batch inference.
Deliverables:
Make use of any of the techniques we covered in this project to train a model and return predicted probabilities for each row of the test set as a DataFrame with columns index (same as your index from the input test df) and malware_score (predicted probabilities).
Complete the train_model_return_scores function in task5.py
import numpy as np
import pandas as pd
def train_model_return_scores(train_df_path,test_df_path) -> pd.DataFrame:
# TODO: Load and preprocess the train and test dfs
# Train a sklearn model using training data at train_df_path
# Use any sklearn model and return the test index and model scores
# TODO: output dataframe should have 2 columns
# index : this should be the row index of the test df
# malware_score : this should be your model's output for the row in the test df
test_scores = pd.DataFrame()
return test_scores
Trending now
This is a popular solution!
Step by step
Solved in 2 steps
