mlp-m-coen6331-neural-networks-assignment1

pdf

School

Concordia University *

*We aren’t endorsed by this school

Course

691

Subject

Electrical Engineering

Date

Apr 3, 2024

Type

pdf

Pages

29

Uploaded by MajorOxidePartridge34

Report
MLP m - Coen6331 Neural Networks-assignment1 Telecommunication Networks (Concordia University) Scan to open on Studocu Studocu is not sponsored or endorsed by any college or university MLP m - Coen6331 Neural Networks-assignment1 Telecommunication Networks (Concordia University) Scan to open on Studocu Studocu is not sponsored or endorsed by any college or university Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Multi-Layer Perceptron Assignment, By Multi-Layer Perceptron Electrical and Computer Engineering Department, Concordia University Winter-2023 Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 2 Abstract In this report I am going to implement a multi-layer perceptron network also known as the feedforward neural network and I am going to provide a trainer for it including backpropagation and stochastic gradient descent. Then after in this document the characteristics of each hyperparameters are reported. Keywords : Neural Networks, Multi-Layer Perceptron Dataset Firstly, I must create the dataset needed for this assignment. Figure 1 shows the data distributions and its boundary regions. There are to Inputs (X1 and X2) and as shown in the figure there are two classes (C1 and C2). The Figure contains two pair of concentric semicircles. In each pair of concentric semicircles I can generate random points with the distance to the center less than the radius of each semicircles. Therefore, I can generate points belonging to each semicircle. Figure 1, data distributions. I use Python 3, and Jupyter Notebook to generate the dataset. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 3 Figure 2, 12000 generated random points in two classes. Red points are belong to C1 and Blue are belong to C2. The small semicircles are shown in lighter color to show that I have the control in the distribution of points. The codes for dataset creation are provided in appendix A. In appendix B the part of code which shuffles the dataset and splits in into train and test sets are provided. MLP Implementation Now I have the train and test sets. Firstly I implemented the MLP from scratch, then after I used Tensorflow Keras to have a more accurate implementation. The From-Scratch MLP and Tensorflow version of it are provided in Appendix C and D respectively. Appendix E provides the part of code for confusion matrix generation. Test scenarios So far, I created the MLP and its trainer is provided in codes. In this section, I provide the possible test scenarios to evaluate each hyperparameters’ characteristics. Table 1 lists the Hyperparameters. In each test I check only one hyperparameter and I leave the others as the Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 4 default value mentioned in the table. My goal is to find an efficient model that gives a high classification accuracy with the smallest size of the model. Hyperparameter Min Max Default Number of Hidden Layers 1 3 1 Number of Neurons [2, 10, 1] [2, 10, 10, 10, 1] [2, 10, 1] Learning Rate 0.1 1 0.1 Batch Size 1 20 5 Activation Function [sigmoid, tanh, relu] sigmoid Epochs 100 500 100 Dataset Manipulation 1,200 points 12,000 points, more points on edges 1,200 points, and %30 of points has been considered as the test set. Training algorithm SGD SGD Stochastic Gradient Descent (SGD) Default Test (Base Test) In this section I am defining the default test. In next sections I compare the results of default test to the other scenarios to show how a hyperparameter can affect the results. Firstly, let’s have this definition as the Table 2. Hyperparameter Default Description Number of Hidden Layers 1 Number of Neurons [2, 10, 1] 2 inputs, 10 neurons in first layer, 1 output neuron (‘0’ represents C1 and ‘1’ for C2. Learning Rate 0.1 Batch Size 5 Activation Function sigmoid Epochs 100 Dataset 1200 points same for all tests except for Dataset Manipulation Figure 3 shows the default dataset with 1200 points, the training and test sets. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 5 Figure 3, a) Dataset, b) Training set, c) Test set. Figure 4 shows a) how the model accuracy and its validation accuracy changed over the epochs, b) correct and miss classifications, and c) the confusion matrix. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 6 Figure 4, Model training and validation accuracy, miss classification points and the confusion matrix. The accuracy of this test was %87, so consider these results as the reference (base) results and I compare the other scenarios with the reference (base) scenario. Number of Hidden Layers and their Neurons In this test I check the affect of number of hidden layers and the neurons in each layer. Firstly, I want to have two hidden layers and 5 neurons in each. I want to check if I have only 10 neurons, how its distribution in two layers can affect the accuracy and the speed of convergence. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 7 Figure 5, Model training and validation accuracy when we have two hidden layers and 5 neurons in each layer. As it is shown inf Figure 5, in comparison to the reference test which converged around 60 th epoch, the increase of hidden layer will result in late converge at around 90 th epoch. After the feeding the test set, I got %89 accuracy which is %2 higher than the reference test, but this increase is not trustable because it might be from the random initialization of weights. Now I want to increase the number of neurons, without increasing the number of epochs and data points. In this test I have three hidden layers with 10 neurons each. Figure 6 show the results of training. As it is illustrated, there is lots of fluctuations in the training and validation accuracy over the epochs. I expected to have a stronger model that can provide a better accuracy but it can provide %59 accuracy after the test. There might be two possible problems, the saturation problem of the sigmoid activation function, and the problem of a large learning rate. In activation function and learning rate scenario I explain the mentioned problems. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 8 Figure 6, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer. Learning Rate Now I try to fix the problem I mentioned at the end of Number of Hidden Layers and their Neurons section. I have three hidden layers with 10 neurons each, 100 epoch for training 1200 points, and the activation function is sigmoid for all of the neurons. I only change the learning rate to reduce the fluctuation shown in figure 6. In this test I want to increase the learning rate to 0.3. Figure 7 shows show the training accuracy increases during the epochs. According to this figure, after the 40 th epoch there is no significant change in the accuracy, but we have a higher accuracy in comparison to previous test. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 9 Figure 7, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and learning rate is 0.3. Now I want to have a larger learning rate at first epochs and gradually reducing it as the epoch number rises. Figure 8 shows the results for this test. As it is shown, we have less fluctuation and our model is gradually learning during the epochs. We have %89 accuracy at the end. Figure 8, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and automatic decay learning rate starting from 0.99. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 10 Activation Function Now I want to change the activation function. Because I know there is a saturation problem with sigmoid. Therefore, I change the activation function to tanh and relu. Figure 9 and 10 show the three hidden layer MLP with 10 neurons in each layer and tanh and relu activation functions respectively. The learning rate for both are 0.1 as the constant value. I got a high accuracy around %98 as expected form such a big model and shown in figure 9 and 10. Therefore, as it was expected, the most problematic issue was the sigmoids saturation with the problem is increasing with the increase in the number of hidden layers. Figure 9, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and static learning rate 0.1 and all the activation functions are Tanh. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 11 Figure 10, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and static learning rate 0.1 and all the activation functions are Relu. Batch Size Back to the roble we had with sigmoid activation function and the three layer MLP with 10 neurons each. The question is, how the increase or decrease in batch size can affect on the training accuracy during the epochs? To answer this question, I firstly test the model with the batch-size = 1; then after I increase it to 10. Figure 11 and 12 shows the results of both changes on batch size. The accuracy after test for the minimum batch size is %92.5 and for the maxim batch size (in our scenario it is 10) is %57. It seams that in our case because of the number of classes (which is two) minimum batch size will not result in forgetting problem. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 12 Figure 11, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=1. Figure 12, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=10. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 13 Epochs Now I want to increase the number of epochs and check if we give more training iteration over the same training data what would be the slope of training accuracy for the problem mentioned in Number of Hidden Layers and their Neurons section. Thus, we have three hidden layers with 10 neurons each, and the learning rate is static set as 0.1. The batch size is 5 and all the activation functions are sigmoid. Figure 13 shows the accuracy over the training in 500 epochs. Figure 13, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=500. After the test, the accuracy of this model for the prediction is %96. At it is shown in figure 13, the fluctuations from 0 to 100 th epoch are the same as Figure 6. Increasing the epochs number sometimes help, but always there is a risk of overfitting on the training data. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 14 Dataset Manipulation In the tested scenarios I understood most the miss classifications are on the corners (the borders witch are shared between two classes). Now I want to test how bias in the data set can change the results. As it is shown in Figure 14, we have a biased dataset with more data points close to the borders. Figure 14, Biased dataset with 1200 points. After the tests as it is shown in Figure the miss classification area has been changed due to the bias. Therefore, I want to increase the number of data points to 12,000. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 15 Figure 15, a) Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=100. B) miss classification area. Figure 16, Biased dataset with 12000 points. As it is expected, more data points (sample) on a simple classification problem, can be considered as a larger epochs because with too many samples we have multiple samples with less deference. Final accuracy is %79 which is higher than we had in Figure 6 (%57). Figure 17 shows the results of this test and its miss classification area. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 16 Figure 17, Over biased dataset with 12000 points, a) Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=100. B) miss classification area. Efficient Model and Conclusion In this test I am using a combination of all the hyperparameters I studied in this report to build an efficient model. Figure 18 shows the model. Figure 18, my efficient model with 3 hidden layer and 3 neurons in each hidden layer. Hyperparameter Default Description Number of Hidden Layers 3 Number of Neurons [2, 3, 3, 3,1] 2 inputs, 1 output neuron (‘0’ represents C1 and ‘1’ for C2. Learning Rate Automatic Start rate=0.5 Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 17 Hyperparameter Default Description Batch Size 1 Activation Function Tanh Epochs 100 Dataset 1200 points same for all tests except for Dataset Manipulation Figure 19 shows the results of this test and its miss classification area. The accuracy of this model after test is %92.5 which shows to get higher accuracy there is no need to increase the size of model and its complexity. With a proper hyperparameter selection we can get a sufficient accuracy with less resource utilization. If we increase the bias of our training set, we also are increasing the biased samples in test set. However, the model which is trained on the biased dataset may provide better accuracy on unbiased dataset. Figure 19, Final results for the efficient model. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 18 Appendix A: Dataset This is the dataset generator code. In this code I provided more flexibility to generate desired number of points in eight different regions. import numpy as np import random import matplotlib.pyplot as plt import math def blongto(cx, cy, r, x,y): d = math.sqrt((x - cx) ** 2 + (y - cy) ** 2 ) ; if d < r: return True return False #Big circle Left r3 = 2 ; C1x1 = [] ; C1x2 = [] ; #Big circle Right r4 = 2 ; C2x1 = [] ; C2x2 = [] ; #Small circle Right r2 = 1 ; C1x1_2 = [] ; C1x2_2 = [] ; #Small circle Left r1 = 1 ; C2x1_2 = [] ; C2x2_2 = [] ; #centers #center Left cx1 = cy1 = 0 ; #center Right cx2 = 0 ; cy2 =- 1 ; #Criticl Circle Left big rc2 = 1.1 ; C1cx1 = [] ; C1cx2 = [] ; #Criticl Circle Left small rc1 = 0.9 ; C2cx1 = [] ; C2cx2 = [] ; #Criticl Circle Right big rc1_2 = 0.9 ; C1cx1_2 = [] ; C1cx2_2 = [] ; #Criticl Circle Right small rc2_2 = 1.1 ; C2cx1_2 = [] ; C2cx2_2 = [] ; count = 0 ; Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 19 nCritical = 2500 nNormal = 1000 max_count = [nNormal , nCritical, nCritical, nNormal, nNormal, nCritical, nCri tical, nNormal] count = [ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ] while ( True ): x = random.uniform( - 2 , 2 ) ; y = random.uniform( - 3 , 2 ) ; if x < 0 : if blongto(cx1,cy1, r3, x,y): if blongto(cx1,cy1, r1, x,y): #small circle left if blongto(cx1,cy1, rc1, x,y) and (count[ 0 ] < max_count[ 0 ]): count[ 0 ] += 1 C2x1_2.append(x) C2x2_2.append(y) elif not blongto(cx1,cy1, rc1, x,y) and (count[ 1 ] < max_count [ 1 ]): #Criticl Circle Left small count[ 1 ] += 1 C2cx1.append(x) C2cx2.append(y) else : #Big circle Left if blongto(cx1,cy1, rc2, x,y) and (count[ 2 ] < max_count[ 2 ]): #Criticl Circle Left small count[ 2 ] += 1 C1cx1.append(x) C1cx2.append(y) elif not blongto(cx1,cy1, rc2, x,y) and (count[ 3 ] < max_cou nt[ 3 ]): count[ 3 ] += 1 C1x1.append(x) C1x2.append(y) else : if blongto(cx2,cy2, r4, x,y): if blongto(cx2,cy2, r2, x,y): #small circle Right if blongto(cx2,cy2, rc1_2, x,y) and (count[ 4 ] < max_count [ 4 ]): count[ 4 ] += 1 C1x1_2.append(x) C1x2_2.append(y) elif not blongto(cx2,cy2, rc1_2, x,y) and (count[ 5 ] < max_co unt[ 5 ]): count[ 5 ] += 1 C1cx1_2.append(x) C1cx2_2.append(y) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 20 else : #Big circle Right if blongto(cx2,cy2, rc2_2, x,y) and (count[ 6 ] < max_count [ 6 ]): count[ 6 ] += 1 C2cx1_2.append(x) C2cx2_2.append(y) elif not blongto(cx2,cy2, rc2_2, x,y) and (count[ 7 ] < max_co unt[ 7 ]): count[ 7 ] += 1 C2x1.append(x) C2x2.append(y) if (np. sum (count) == np. sum (max_count)): break s = 1 plt.scatter(C1x1, C1x2, color = 'red' , s = s) plt.scatter(C1x1_2, C1x2_2, color = 'red' , s = s) plt.scatter(C2x1, C2x2, color = 'blue' , s = s) plt.scatter(C2x1_2, C2x2_2, color = 'blue' , s = s) #critical Circles plt.scatter(C1cx1, C1cx2, color = [ 0.7 , 0 , 0 ], s = s) plt.scatter(C2cx1, C2cx2, color = [ 0 , 0 , 0.7 ], s = s) plt.scatter(C1cx1_2, C1cx2_2, color = [ 0.7 , 0 , 0 ], s = s) plt.scatter(C2cx1_2, C2cx2_2, color = [ 0 , 0 , 0.7 ], s = s) plt.show() data = [] for i in range ( len (C1x1)): c = [C1x1[i], C1x2[i], 0 ] data.append(c) for i in range ( len (C2x1)): c = [C2x1[i], C2x2[i], 1 ] data.append(c) for i in range ( len (C1x1_2)): c = [C1x1_2[i], C1x2_2[i], 0 ] data.append(c) for i in range ( len (C2x1_2)): c = [C2x1_2[i], C2x2_2[i], 1 ] data.append(c) #criticals for i in range ( len (C2cx1)): c = [C2cx1[i], C2cx2[i], 1 ] data.append(c) for i in range ( len (C1cx1)): c = [C1cx1[i], C1cx2[i], 0 ] Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 21 data.append(c) for i in range ( len (C2cx1_2)): c = [C2cx1_2[i], C2cx2_2[i], 1 ] data.append(c) for i in range ( len (C1cx1_2)): c = [C1cx1_2[i], C1cx2_2[i], 0 ] data.append(c) from sklearn.utils import shuffle print (np.shape(data)) print ( type (np.array(data))) data2 = np.array(data) Appendix B: Train and Test sets This part of code generates the test and train set. Firstly I shuffle the dataset then I create the test and train sets based on test_size parameter. np.random.shuffle(data2) test_size = 0.3 ; total = np.shape(data)[ 0 ] train_index = int (total * ( 1 - test_size)) train_inputs = data2[ 0 :train_index, 0 : 2 ] test_inputs = data2[train_index: , 0 : 2 ] train_outputs = data2[ 0 :train_index, 2 : 4 ] test_outputs = data2[ train_index:, 2 : 4 ] plt.scatter(train_inputs[:, 0 ], train_inputs[:, 1 ], color = [ 0 , 0 , 0.7 ], s = s) plt.show() Appendix C: From-Scratch MLP import numpy as np from random import random Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 22 class MLP: def __init__ ( self , num_inputs = 3 , num_hidden = [ 3 , 5 ], num_outputs = 2 ): self .num_inputs = num_inputs self .num_hidden = num_hidden self .num_outputs = num_outputs layers = [ self .num_inputs] + self .num_hidden + [ self .num_outputs] #initiate random weights self .weights = [] for i in range ( len (layers) - 1 ): w = np.random.rand(layers[i], layers[i + 1 ]) self .weights.append(w) self .activations = [] for i in range ( len (layers)): a = np.zeros(layers[i]) self .activations.append(a) self .derivatives = [] for i in range ( len (layers) - 1 ): d = np.zeros((layers[i], layers[i + 1 ])) self .derivatives.append(d) def forward_propagate( self , inputs): activations = inputs self .activations[ 0 ] = inputs for i, w in enumerate ( self .weights): #calc the net inputs net_inputs = np.dot(activations, w) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 23 #calc the activations activations = self ._sigmoid(net_inputs) self .activations[i + 1 ] = activations return activations def _sigmoid( self , x): return 1 / ( 1 + np.exp( - x)) def back_prop( self , error, verbose = False ): #dE/dW_i = (y - a_[i+1]) s'(h_[i+1])) a_i #s'(h_[i+1]) = s(h_[i+1])(1- s(h_[i+1])) #s(h_[i+1]) = a_[i+1] #dE/dW_[i-1] = (y - a_[i+1]) s'(h_[i+1])) W_i s'(h_i) a_[i-1] for i in reversed ( range ( len ( self .derivatives))): activations = self .activations[i + 1 ] delta = error * self ._sigmoid_derivative(activations) # ndarray([0.1, 0.2])--> ndarray([[0.1 , 0.2]]) delta_reshaped = delta.reshape(delta.shape[ 0 ], - 1 ).T current_activations = self .activations[i] # ndarray([0.1, 0.2])-- > ndarray([[0.1], [0.2]]) current_activations_reshaped = current_activations.reshape(current_activations.shape[ 0 ], - 1 ) self .derivatives[i] = np.dot(current_activations_reshaped, delta_reshaped) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 24 error = np.dot(delta, self .weights[i].T) if verbose: print ( "Derivatives for W{}: {}" . format (i, self .derivatives[i])) return error def gradient_desceent( self , learning_rate): for i in range ( len ( self .weights)): weights = self .weights[i] #print("Original W{} {}".format(i, weights)) derivatives = self .derivatives[i] weights += derivatives * learning_rate #print("Updated W{} {}".format(i, weights)) def train( self , inputs, targets, epochs, learning_rate): for i in range (epochs): sum_error = 0 for input , target in zip (inputs, targets): #pertform forward prop output = self .forward_propagate( input ) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 25 #calc the Error error = target - output #back prop self .back_prop(error, verbose = False ) #APPLY gradient_descent self .gradient_desceent(learning_rate) sum_error += self ._mse(target, output) # report the error print ( "Error: {} at Epoch {}" . format (sum_error / len (inputs), i)) def _mse( self , target, output): return np.average((target - output) ** 2 ) def _sigmoid_derivative( self , x): return x * ( 1.0 - x) if __name__ == "__main__" : # create a Multilayer Perceptron with one hidden layer mlp = MLP( 2 , [ 10 , 3 ], 2 ) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 26 # train network mlp.train(train_inputs, train_outputs, 100 , 0.1 ) # get a prediction hit = 0 ; for i in range (np.shape(test_inputs)[ 0 ]): y = test_outputs[ 0 ] output = mlp.forward_propagate(test_inputs[i]) if (output[ 0 ] > output[ 1 ]): if (y[ 0 ] == 1 ): hit += 1 else : if (y[ 1 ] == 1 ): hit += 1 print (hit / np.shape(test_inputs)[ 0 ]) Appendix D: Tensorflow Keras MLP import tensorflow as tf from sklearn.model_selection import train_test_split import numpy as np from random import random # create a dataset with 2000 samples x_train = train_inputs x_test = test_inputs y_train = train_outputs y_test = test_outputs # build model with 3 layers: 2 -> 5 -> 1 Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 27 model = tf.keras.models.Sequential([ tf.keras.layers.Dense( 10 , input_dim = 2 , activation = "relu" ), #relu, tanh, sigmoid or tf.keras.layers.LeakyReLU(alpha=0.01) tf.keras.layers.Dense( 10 , activation = "relu" ), tf.keras.layers.Dense( 1 , activation = "relu" ) ]) tf.keras.layers.Lea # choose optimiser optimizer = tf.keras.optimizers.SGD(learning_rate = 0.087 ) # compile model model. compile (optimizer = optimizer, loss = 'mse' ) # train model model.fit(x_train, y_train, epochs = 100 , batch_size= 5 ) # evaluate model on test set print ( "\nEvaluation on the test set:" ) model.evaluate(x_test, y_test, verbose = 2 ) hit = 0 ; miss = 0 missC = [] hitC = [] output = model.predict(x_test) for i in range (np.shape(test_inputs)[ 0 ]): if (output[i] > 0.5 ): output[i] = 1 else : output[i] = 0 for i in range (np.shape(test_inputs)[ 0 ]): if (y_test[i] == output[i]): Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help
MLP ASSIGNMENT, BY 28 hit += 1 hitC.append(test_inputs[i]) else : miss += 1 missC.append(test_inputs[i]) print (hit,miss) print (hit / np.shape(test_inputs)[ 0 ]) hitC = np.array(hitC) missC = np.array(missC) plt.scatter(hitC[:, 0 ], hitC[:, 1 ], color = [ 1 , 0 , 0 ], s = s) plt.scatter(missC[:, 0 ], missC[:, 1 ], color = [ 0 , 0 , 0.5 ], s = 5 ) plt.show() Appendix E: confusion matrix generation from sklearn.metrics import ConfusionMatrixDisplay from sklearn.metrics import confusion_matrix import sklearn.metrics as metrics labels = [ "C1" , "C2" ] cm = confusion_matrix(y_test, output) disp = ConfusionMatrixDisplay(confusion_matrix = cm, display_labels = labels) disp.plot(cmap = plt.cm.Blues) plt.show() Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com) lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help

Related Documents

William Spann - Calculating Ohm's Law practice (google sheet) _SELF GRADING_.xlsx
Homework 1.pdf
Julius Parks - Human and Physical Regions.pdf
assignment2_2024.docx
Lab 4 Report Sheet (1).pdf
hw3.pdf
JavaElectricBillLabReport.docx
ECE_161B_HW4_IFIR.docx
EET230_Lab5_Taylor_Jason.docx
APES_personalenergyaudit.docx
ECE_161A_Project_2023.docx
Hw movie 2.docx
Related Questions
No paper
I need the expert solution and a detailed explanation of each step in the solution.
De fine time constant (I) for (R-L) cct.
HW1: Drive the Moore model state diagrams for a sequence 101 detector and then check it for possible state reduction. Hint: draw the state diagram (Moore model), convert the state diagram to state table first and then use the implication chart to find the reduced state table. mra
What theoretical transfer function models happen with a theoretical hvac system heating and cooling a home
Please review the images.
(B): Design a Moore FSM to detect the sequence (00) using T-FFs. Q4/ (A): Design a Mealy FSM for odd parity generator using JK-FFs.
Write a PIC16F877A program that flash ON the 8-LED's connected to port-B by using two switches connected to port-D (Do & D₁) as shown in figure below, according to the following scenarios: (Hint: Use 500ms delay for each case with 4MHz frequency) 1. When Do=1 then B₁,B3,B7 are ON. 2. When Do 0 then Bo,B2, B4, B5, B6 are ON. 3. When D=1 then B4, B5, B6, B7 are ON. 4. When D₁=0 then B0,B1,B2,B3 are ON. U1 OSC1/CLION 33 OSC2/CLKOUT T 34 5 U2 20 10 35 3 18 10 - 30 -ou-un RB2 2 RA1/AN1 PA3/AN3VREF+ RA4/TOCKIC1OUT RACIANO 0RB3PGM 37 33 4 17 5 10 RB4 39 RA2/ANZ/VREF-CVREF RB5 15 39 7 14 RB6PGC 40 13 RB7/PGO RAS/AN4/SS/C2OUT + 15 RCOT1OSOTICK 10 RECANARD RE1/AN/R REZ/ANTICS ✓ MCLR/Vpp/THV RC2/CCP1 RC3/SCK/SCL RCA/SOISDA RC5/S00 RCB/TX/CK 10 RC1/T10SUCCP2 17 LED-BARGRAPH-RED RC7/RX/DT ROOPSPO RD1/PSP1 R02/PSP2 RD3/PSP3 ROUPSP4 RD5PSP5 ROOPSPO RO7/PSP7 PIC16F877A +5V R2 100R R1 100R
0.4/ For each pair of second-order system specifications that follow, find the location of the second-order pair of poles. a. %OS = 12%; Ts = 0.6 second b. %OS = 10%; Tp %3D = 5 seconds %3D c. Ts =7 seconds; Tp=3 seconds
For the system shown: a. Get the equations b. Simplify and determine C/R c. Draw the signal flow diagram for this system. H3 Z3 m1 R E E1 G1 G2 G3 G4 m n H2 Z1 22 H1
SEE MORE QUESTIONS
Recommended textbooks for you
Text book image
Introductory Circuit Analysis (13th Edition)
Electrical Engineering
ISBN:9780133923605
Author:Robert L. Boylestad
Publisher:PEARSON
Text book image
Delmar's Standard Textbook Of Electricity
Electrical Engineering
ISBN:9781337900348
Author:Stephen L. Herman
Publisher:Cengage Learning
Text book image
Programmable Logic Controllers
Electrical Engineering
ISBN:9780073373843
Author:Frank D. Petruzella
Publisher:McGraw-Hill Education
Text book image
Fundamentals of Electric Circuits
Electrical Engineering
ISBN:9780078028229
Author:Charles K Alexander, Matthew Sadiku
Publisher:McGraw-Hill Education
Text book image
Electric Circuits. (11th Edition)
Electrical Engineering
ISBN:9780134746968
Author:James W. Nilsson, Susan Riedel
Publisher:PEARSON
Text book image
Engineering Electromagnetics
Electrical Engineering
ISBN:9780078028151
Author:Hayt, William H. (william Hart), Jr, BUCK, John A.
Publisher:Mcgraw-hill Education,
SEE MORE TEXTBOOKS
Related Questions
Recommended textbooks for you
Text book image
Introductory Circuit Analysis (13th Edition)
Electrical Engineering
ISBN:9780133923605
Author:Robert L. Boylestad
Publisher:PEARSON
Text book image
Delmar's Standard Textbook Of Electricity
Electrical Engineering
ISBN:9781337900348
Author:Stephen L. Herman
Publisher:Cengage Learning
Text book image
Programmable Logic Controllers
Electrical Engineering
ISBN:9780073373843
Author:Frank D. Petruzella
Publisher:McGraw-Hill Education
Text book image
Fundamentals of Electric Circuits
Electrical Engineering
ISBN:9780078028229
Author:Charles K Alexander, Matthew Sadiku
Publisher:McGraw-Hill Education
Text book image
Electric Circuits. (11th Edition)
Electrical Engineering
ISBN:9780134746968
Author:James W. Nilsson, Susan Riedel
Publisher:PEARSON
Text book image
Engineering Electromagnetics
Electrical Engineering
ISBN:9780078028151
Author:Hayt, William H. (william Hart), Jr, BUCK, John A.
Publisher:Mcgraw-hill Education,
SEE MORE TEXTBOOKS