TML_Assignment_3

CIS 6261: Trustworthy Machine Learning (Spring 2023) Homework 3 — Privacy Attacks on Machine Learning Models Name: Your Name Here April 10, 2023 This is an individual assignment. Academic integrity violations (i.e., cheating, plagiarism) will be reported to SCCR! The official CISE policy recommended for such offenses is a course grade of E. Additional sanctions may be imposed by SCCR such as marks on your permanent educational transcripts, dismissal or expulsion. Reminder of the Honor Pledge: On all work submitted for credit by Students at the University of Florida, the following pledge is either required or implied: “On my honor, I have neither given nor received unauthorized aid in doing this assignment.” Instructions Please read the instructions and questions carefully. Write your answers directly in the space provided. Compile the tex document and hand in the resulting PDF. In this assignment you will implement and evaluate several membership inference attacks in Python. Use the code skeleton provided and submit the completed source file(s) alongside with the PDF. 1 Assignment Files The assignment archive contains the following Python source files: • hw.py . This file is the main assignment source file. • nets.py . This file defines the neural network architectures and some useful related functions. • attacks.py . This file contains attack code used in the assignment. Note: You are encouraged to take a look at the provided files. This may help you successfully complete the assignment. 1 You should use Python3 with Tensorflow 2. You may use HiPerGator or your own system. If you use your own system you may need to install the required packages (e.g., numpy, scikit-learn, tensorflow, etc.). This assignment can be done with or without GPUs. 1

Problem 1: Training Neural Networks (20 pts) For this problem, you will train neural networks to do MNIST classification. You will get familiar with the provided code skeleton and answer some questions about what you observe when running the code. To run the code for this problem, use the following command. python3 hw.py problem1 <nn_desc> <num_epoch> Here <nn desc> is a neural network description string (no whitespaces). It can take two forms: simple,<num hidden>,<l2 reg const> or deep . The latter specifies the deep neural network architecture (see get deeper classifier() in nets.py for details), whereas the former specifies a simple neural network architecture (see get simple classifier() in nets.py for details) with one hidden layer with <num hidden> neurons and an L 2 regularization constant of <l2 reg const> . Also, <num epoch> is the number of epoch to train for. For example, suppose you run the following command. python3 hw.py problem1 simple,32,0.001 50 This will train the target model on 2000 MNIST data records (i.e., images) for 50 epochs. The target model architecture is a neural network with a single hidden layer of 32 neurons which uses L 2 regulariza- tion with a constant of 0 . 001. 2 (The loss function is the categorical cross entropy.) 1. (5 pts) Run the code using the command provided above. What is the training accuracy? What is the test accuracy? How overfitted is the target model you just trained? Training accuracy - 90.0% Test accuracy - 82.8% The difference is 7% between the accuracies which is not too large.The Model is not overfitted 2. (10 pts) Run the code to train the target model while varying the number of training epochs (from 10 to 1000), the number of hidden layer neurons (from 32 to 1024), and the regularization constant (from 0.0 to 0.01). What do you observe? For what combination of parameters is the train/test accuracy the highest? For each parameter you varied, explain its impact on accuracy and overfitting. 2 By default, for problem 1, the code will provide detailed output about the training process and the accuracy of the target model. 2

Problem 2: Implementing the Shokri et al. Attack (30 pts) For this problem you will implement the Shokri et al. [ 1 ] attack. Put your code in the shokri attack models() function which is located in attacks.py . The companion function do shokri attack() is already pro- vided so you do not have to implement it. Refer to the course slides and/or the paper [ 1 ] for instructions on how to implement the attack. Comments in the provided code skeleton are here to guide your implementation. To run the code for this problem, use the following. python3 hw.py problem2 <nn_desc> <num_epoch> <num_shadow> <attack_model> Here <num shadow> is the number of shadow models to use and <attack model> is a string denoting the attack model type (scikit-learn). See the code for options. Answer the following questions. 1. (10 pts) Once you have implemented the attack, run the following command: python3 hw.py problem2 simple,1024,0.0 50 10 LR Note: you may need to re-train the model (see problem 1). What is the accuracy of the attack? What is the advantage? Your answer here. 2. (10 pts) Run the attack on the most overfitted target model (according to the parameters you found in the previous problems and your overfitting metric). Also run the attack on the least overfitted but best performing target model. Compare the attack accuracy and advantage in both cases. What do you notice? According to your experiments what are the factors that lead to attack success? (Justify your answer.) Your answer here. 3. (10 pts) Now vary the attack model type. What do you notice? What are the best/worst performing attack model types? Your answer here. 4

Problem 3: More membership inference attacks (30 pts) For this problem you will implement three more attacks. • Loss attack ( do loss attack() ): for this attack you are given the mean and standard deviation of the target model’s training loss and testing loss. You can assume that the loss follows a Gaussian distribution. The attack works as follows: given a target record, measure its loss on the target model and then decide (IN or OUT) based on which loss distribution the sample most likely comes from. • Loss attack2 ( do loss attack2() ): for this attack you are given only the mean and standard deviation of the target model’s training loss. In addition, the attack function takes a threshold parameter. Again, you can assume that the loss follows a Gaussian distribution. The attack works as follows: given a target record, measure its loss on the target model and then decide (IN or OUT) based on whether the probability of obtaining a loss value as extreme or less extreme than that is lower than the threshold. • Posterior attack ( do posterior attack() ): for this attack you only have the ability to query the target model. You are also given a threshold parameter. The attack works as follows: given a target record, query it on the target model to obtain its posterior (i.e., the predicted probability over the true class label) and then decide (IN or OUT) based on whether the posterior is greater than the threshold. Implement all three attacks and answer the following questions. 1. (5 pts) Pick a set of parameters. What is the attack accuracy and advantage of the loss attack? Specify the command you ran! Your answer here. 2. (10 pts) Loss attack2 takes a threshold parameter. What is the optimal threshold value? Justify your answer. (Hint: you can write some code to find out experimentally.) Your answer here. 3. (10 pts) The posterior attack takes a threshold parameter. What is the optimal threshold value? Justify your answer. (Hint: you can write some code to find out experimentally.) Your answer here. 4. (5 pts) Which of the three attacks performs best? Your answer here. 5

References [1] Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP) (2017), IEEE, pp. 3–18. 7