mlp-m-coen6331-neural-networks-assignment1
pdf
keyboard_arrow_up
School
Concordia University *
*We aren’t endorsed by this school
Course
691
Subject
Electrical Engineering
Date
Apr 3, 2024
Type
Pages
29
Uploaded by MajorOxidePartridge34
MLP m - Coen6331 Neural Networks-assignment1
Telecommunication Networks (Concordia University)
Scan to open on Studocu
Studocu is not sponsored or endorsed by any college or university
MLP m - Coen6331 Neural Networks-assignment1
Telecommunication Networks (Concordia University)
Scan to open on Studocu
Studocu is not sponsored or endorsed by any college or university
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Multi-Layer Perceptron Assignment, By Multi-Layer Perceptron
Electrical and Computer Engineering Department, Concordia University
Winter-2023
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 2
Abstract
In this report I am going to implement a multi-layer perceptron network also known as the feedforward neural network and I am going to provide a trainer for it including backpropagation and stochastic gradient descent. Then after in this document the characteristics of each hyperparameters are reported. Keywords
: Neural Networks, Multi-Layer Perceptron
Dataset
Firstly, I must create the dataset needed for this assignment. Figure 1
shows the data distributions and its boundary regions. There are to Inputs (X1 and X2) and as shown in the figure there are two classes (C1 and C2). The Figure contains two pair of concentric semicircles. In each pair of concentric semicircles I can generate random points with the distance to the center less than the radius of each semicircles. Therefore, I can generate points belonging to each semicircle.
Figure 1, data distributions.
I use Python 3, and Jupyter Notebook to generate the dataset.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 3
Figure 2, 12000 generated random points in two classes. Red points are belong to C1 and Blue are belong to C2. The small semicircles are shown in lighter color to show that I have the control in the distribution of points.
The codes for dataset creation are provided in appendix A. In appendix B the part of code which shuffles the dataset and splits in into train and test sets are provided.
MLP Implementation
Now I have the train and test sets. Firstly I implemented the MLP from scratch, then after I used Tensorflow Keras to have a more accurate implementation. The From-Scratch MLP and Tensorflow version of it are provided in Appendix C and D respectively. Appendix E provides the part of code for confusion matrix generation.
Test scenarios
So far, I created the MLP and its trainer is provided in codes. In this section, I provide the possible test scenarios to evaluate each hyperparameters’ characteristics. Table 1
lists the Hyperparameters. In each test I check only one hyperparameter and I leave the others as the Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 4
default value mentioned in the table. My goal is to find an efficient model that gives a high classification accuracy with the smallest size of the model.
Hyperparameter
Min
Max
Default
Number of Hidden Layers
1
3
1
Number of Neurons
[2, 10, 1]
[2, 10, 10, 10, 1]
[2, 10, 1]
Learning Rate
0.1
1
0.1
Batch Size
1
20
5
Activation Function
[sigmoid, tanh, relu]
sigmoid
Epochs
100
500
100
Dataset Manipulation
1,200 points
12,000 points, more points on edges
1,200 points, and %30 of points has been considered as the test set.
Training algorithm
SGD
SGD
Stochastic Gradient Descent (SGD)
Default Test (Base Test)
In this section I am defining the default test. In next sections I compare the results of default test to the other scenarios to show how a hyperparameter can affect the results. Firstly, let’s have this definition as the Table 2.
Hyperparameter
Default
Description
Number of Hidden Layers
1
Number of Neurons
[2, 10, 1]
2 inputs, 10 neurons in first layer, 1 output neuron (‘0’ represents C1 and ‘1’ for C2.
Learning Rate
0.1
Batch Size
5
Activation Function
sigmoid
Epochs
100
Dataset
1200 points
same for all tests except for Dataset Manipulation
Figure 3 shows the default dataset with 1200 points, the training and test sets.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 5
Figure 3, a) Dataset, b) Training set, c) Test set.
Figure 4 shows a) how the model accuracy and its validation accuracy changed over the epochs, b) correct and miss classifications, and c) the confusion matrix. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 6
Figure 4, Model training and validation accuracy, miss classification points and the confusion matrix.
The accuracy of this test was %87, so consider these results as the reference (base) results and I compare the other scenarios with the reference (base) scenario.
Number of Hidden Layers and their Neurons
In this test I check the affect of number of hidden layers and the neurons in each layer. Firstly, I want to have two hidden layers and 5 neurons in each. I want to check if I have only 10 neurons, how its distribution in two layers can affect the accuracy and the speed of convergence.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 7
Figure 5, Model training and validation accuracy when we have two hidden layers and 5 neurons in each layer.
As it is shown inf Figure 5, in comparison to the reference test which converged around 60
th
epoch, the increase of hidden layer will result in late converge at around 90
th
epoch. After the feeding the test set, I got %89 accuracy which is %2 higher than the reference test, but this increase is not trustable because it might be from the random initialization of weights.
Now I want to increase the number of neurons, without increasing the number of epochs and data points. In this test I have three hidden layers with 10 neurons each. Figure 6 show the results of training. As it is illustrated, there is lots of fluctuations in the training and validation accuracy over the epochs. I expected to have a stronger model that can provide a better accuracy but it can provide %59 accuracy after the test. There might be two possible problems, the saturation problem of the sigmoid activation function, and the problem of a large learning rate. In activation function and learning rate scenario I explain the mentioned problems.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 8
Figure 6, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer.
Learning Rate
Now I try to fix the problem I mentioned at the end of Number of Hidden Layers and their Neurons section. I have three hidden layers with 10 neurons each, 100 epoch for training 1200 points, and the activation function is sigmoid for all of the neurons. I only change the learning rate to reduce the fluctuation shown in figure 6. In this test I want to increase the learning rate to 0.3. Figure 7 shows show the training accuracy increases during the epochs. According to this figure, after the 40
th
epoch there is no significant change in the accuracy, but we have a higher accuracy in comparison to previous test. Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 9
Figure 7, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and learning rate is 0.3.
Now I want to have a larger learning rate at first epochs and gradually reducing it as the epoch number rises. Figure 8 shows the results for this test. As it is shown, we have less fluctuation and our model is gradually learning during the epochs. We have %89 accuracy at the end.
Figure 8, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and automatic decay learning rate starting from 0.99.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 10
Activation Function
Now I want to change the activation function. Because I know there is a saturation problem with sigmoid. Therefore, I change the activation function to tanh and relu. Figure 9 and 10 show the three hidden layer MLP with 10 neurons in each layer and tanh and relu activation functions respectively. The learning rate for both are 0.1 as the constant value.
I got a high accuracy around %98 as expected form such a big model and shown in figure 9 and 10. Therefore, as it was expected, the most problematic issue was the sigmoids saturation with the problem is increasing with the increase in the number of hidden layers.
Figure 9, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and static learning rate 0.1 and all the activation functions are Tanh.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 11
Figure 10, Model training and validation accuracy when we have three hidden layers and 10 neurons in each layer and static learning rate 0.1 and all the activation functions are Relu.
Batch Size
Back to the roble we had with sigmoid activation function and the three layer MLP with 10 neurons each. The question is, how the increase or decrease in batch size can affect on the training accuracy during the epochs? To answer this question, I firstly test the model with the batch-size = 1; then after I increase it to 10. Figure 11 and 12 shows the results of both changes on batch size. The accuracy after test for the minimum batch size is %92.5 and for the maxim batch size (in our scenario it is 10) is %57. It seams that in our case because of the number of classes (which is two) minimum batch size will not result in forgetting problem.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 12
Figure 11, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=1.
Figure 12, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=10.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 13
Epochs
Now I want to increase the number of epochs and check if we give more training iteration over the same training data what would be the slope of training accuracy for the problem mentioned in Number of Hidden Layers and their Neurons section. Thus, we have three hidden layers with 10 neurons each, and the learning rate is static set as 0.1. The batch size is 5 and all the activation functions are sigmoid. Figure 13 shows the accuracy over the training in 500 epochs. Figure 13, Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=500.
After the test, the accuracy of this model for the prediction is %96. At it is shown in figure 13, the fluctuations from 0 to 100
th
epoch are the same as Figure 6. Increasing the epochs number sometimes help, but always there is a risk of overfitting on the training data.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 14
Dataset Manipulation
In the tested scenarios I understood most the miss classifications are on the corners (the borders witch are shared between two classes). Now I want to test how bias in the data set can change the results. As it is shown in Figure 14, we have a biased dataset with more data points close to the borders.
Figure 14, Biased dataset with 1200 points.
After the tests as it is shown in Figure the miss classification area has been changed due to the bias. Therefore, I want to increase the number of data points to 12,000.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 15
Figure 15, a) Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=100. B) miss classification area.
Figure 16, Biased dataset with 12000 points.
As it is expected, more data points (sample) on a simple classification problem, can be considered as a larger epochs because with too many samples we have multiple samples with less deference. Final accuracy is %79 which is higher than we had in Figure 6 (%57). Figure 17 shows the results of this test and its miss classification area.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 16
Figure 17, Over biased dataset with 12000 points, a) Model training and validation accuracy when we have three hidden layers, 10 neurons in each layer, static learning rate=0.1, all the activation functions are Sigmoid and the batch_size=5, epochs=100. B) miss classification area.
Efficient Model and Conclusion
In this test I am using a combination of all the hyperparameters I studied in this report to build an efficient model. Figure 18 shows the model.
Figure 18, my efficient model with 3 hidden layer and 3 neurons in each hidden layer.
Hyperparameter
Default
Description
Number of Hidden Layers
3
Number of Neurons
[2, 3, 3, 3,1]
2 inputs, 1 output neuron (‘0’ represents C1 and ‘1’ for C2.
Learning Rate
Automatic
Start rate=0.5
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 17
Hyperparameter
Default
Description
Batch Size
1
Activation Function
Tanh
Epochs
100
Dataset
1200 points
same for all tests except for Dataset Manipulation
Figure 19 shows the results of this test and its miss classification area. The accuracy of this model after test is %92.5 which shows to get higher accuracy there is no need to increase the size of model and its complexity. With a proper hyperparameter selection we can get a sufficient accuracy with less resource utilization. If we increase the bias of our training set, we also are increasing the biased samples in test set. However, the model which is trained on the biased dataset may provide better accuracy on unbiased dataset.
Figure 19, Final results for the efficient model.
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 18
Appendix A: Dataset
This is the dataset generator code. In this code I provided more flexibility to generate desired number of points in eight different regions.
import
numpy as
np import
random import
matplotlib.pyplot as
plt import
math def
blongto(cx, cy, r, x,y): d = math.sqrt((x
-
cx)
**
2 + (y
-
cy)
**
2
)
; if
d < r: return
True return
False #Big circle Left
r3
= 2
; C1x1 =
[]
; C1x2 =
[]
; #Big circle Right
r4
= 2
; C2x1 =
[]
; C2x2 =
[]
; #Small circle Right
r2
= 1
; C1x1_2 =
[]
; C1x2_2 =
[]
; #Small circle Left
r1
= 1
; C2x1_2 =
[]
; C2x2_2 =
[]
; #centers
#center Left
cx1
=
cy1
=
0
; #center Right
cx2
=
0
;
cy2
=-
1
; #Criticl Circle Left big
rc2
=
1.1
; C1cx1 =
[]
; C1cx2 =
[]
; #Criticl Circle Left small
rc1
=
0.9
; C2cx1 =
[]
; C2cx2 =
[]
; #Criticl Circle Right big
rc1_2
=
0.9
; C1cx1_2 =
[]
; C1cx2_2 =
[]
; #Criticl Circle Right small
rc2_2
=
1.1
; C2cx1_2 =
[]
; C2cx2_2 =
[]
; count =
0
; Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 19
nCritical
=
2500 nNormal = 1000 max_count
= [nNormal , nCritical, nCritical, nNormal, nNormal, nCritical, nCri
tical, nNormal] count
= [
0 , 0
, 0
, 0 , 0
, 0
, 0 , 0
] while
(
True
): x = random.uniform(
-
2
,
2
)
; y = random.uniform(
-
3
,
2
)
; if
x
<
0 : if
blongto(cx1,cy1, r3, x,y): if
blongto(cx1,cy1, r1, x,y): #small circle left
if
blongto(cx1,cy1, rc1, x,y) and
(count[
0
] < max_count[
0
]): count[
0
] +=
1 C2x1_2.append(x) C2x2_2.append(y) elif
not
blongto(cx1,cy1, rc1, x,y) and
(count[
1
] < max_count
[
1
]): #Criticl Circle Left small
count[
1
] +=
1 C2cx1.append(x) C2cx2.append(y) else
: #Big circle Left
if
blongto(cx1,cy1, rc2, x,y) and
(count[
2
] < max_count[
2
]): #Criticl Circle Left small
count[
2
] +=
1 C1cx1.append(x) C1cx2.append(y) elif
not
blongto(cx1,cy1, rc2, x,y) and
(count[
3
] < max_cou
nt[
3
]): count[
3
] +=
1 C1x1.append(x) C1x2.append(y) else
: if
blongto(cx2,cy2, r4, x,y): if
blongto(cx2,cy2, r2, x,y): #small circle Right
if
blongto(cx2,cy2, rc1_2, x,y) and
(count[
4
] < max_count
[
4
]): count[
4
] +=
1 C1x1_2.append(x) C1x2_2.append(y) elif
not
blongto(cx2,cy2, rc1_2, x,y) and
(count[
5
] < max_co
unt[
5
]): count[
5
] +=
1 C1cx1_2.append(x) C1cx2_2.append(y) Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
MLP ASSIGNMENT, BY 20
else
: #Big circle Right
if
blongto(cx2,cy2, rc2_2, x,y) and
(count[
6
] < max_count
[
6
]): count[
6
] +=
1 C2cx1_2.append(x) C2cx2_2.append(y) elif
not
blongto(cx2,cy2, rc2_2, x,y) and
(count[
7
] < max_co
unt[
7
]): count[
7
] +=
1 C2x1.append(x) C2x2.append(y) if
(np.
sum
(count)
==
np.
sum
(max_count)): break
s = 1 plt.scatter(C1x1, C1x2, color
=
'red'
, s
=
s) plt.scatter(C1x1_2, C1x2_2, color
=
'red'
, s
=
s) plt.scatter(C2x1, C2x2, color
=
'blue'
, s
=
s) plt.scatter(C2x1_2, C2x2_2, color
=
'blue'
, s
=
s) #critical Circles
plt.scatter(C1cx1, C1cx2, color
=
[
0.7
,
0
,
0
], s
=
s) plt.scatter(C2cx1, C2cx2, color
=
[
0
,
0
,
0.7
], s
=
s) plt.scatter(C1cx1_2, C1cx2_2, color
=
[
0.7
,
0
,
0
], s
=
s) plt.scatter(C2cx1_2, C2cx2_2, color
=
[
0
,
0
,
0.7
], s
=
s) plt.show() data
=
[] for
i in
range
(
len
(C1x1)): c = [C1x1[i], C1x2[i], 0
] data.append(c) for
i in
range
(
len
(C2x1)): c = [C2x1[i], C2x2[i], 1
] data.append(c) for
i in
range
(
len
(C1x1_2)): c = [C1x1_2[i], C1x2_2[i], 0
] data.append(c) for
i in
range
(
len
(C2x1_2)): c = [C2x1_2[i], C2x2_2[i], 1
] data.append(c) #criticals
for
i in
range
(
len
(C2cx1)): c = [C2cx1[i], C2cx2[i], 1
] data.append(c) for
i in
range
(
len
(C1cx1)): c = [C1cx1[i], C1cx2[i], 0
] Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 21
data.append(c) for
i in
range
(
len
(C2cx1_2)): c = [C2cx1_2[i], C2cx2_2[i], 1
] data.append(c) for
i in
range
(
len
(C1cx1_2)): c = [C1cx1_2[i], C1cx2_2[i], 0
] data.append(c) from
sklearn.utils import
shuffle print
(np.shape(data)) print
(
type
(np.array(data))) data2
=
np.array(data) Appendix B: Train and Test sets
This part of code generates the test and train set. Firstly I shuffle the dataset then I create the test and train sets based on test_size parameter.
np.random.shuffle(data2)
test_size = 0.3
;
total = np.shape(data)[
0
]
train_index = int
(total * (
1
-
test_size))
train_inputs = data2[
0
:train_index,
0
:
2
]
test_inputs = data2[train_index: ,
0
:
2
]
train_outputs = data2[
0
:train_index,
2
:
4
]
test_outputs = data2[ train_index:,
2
:
4
]
plt.scatter(train_inputs[:,
0
], train_inputs[:,
1
], color
=
[
0
,
0
,
0.7
], s
=
s)
plt.show()
Appendix C: From-Scratch MLP
import
numpy as
np
from
random import
random
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 22
class
MLP:
def
__init__
(
self
, num_inputs
=
3
, num_hidden
=
[
3
, 5
], num_outputs
=
2
):
self
.num_inputs = num_inputs
self
.num_hidden = num_hidden
self
.num_outputs = num_outputs
layers = [
self
.num_inputs] + self
.num_hidden + [
self
.num_outputs]
#initiate random weights
self
.weights = []
for
i in
range
(
len
(layers)
-
1
):
w = np.random.rand(layers[i], layers[i
+
1
])
self
.weights.append(w)
self
.activations = []
for
i in
range
(
len
(layers)):
a = np.zeros(layers[i])
self
.activations.append(a)
self
.derivatives = []
for
i in
range
(
len
(layers) -
1
):
d = np.zeros((layers[i], layers[i + 1
]))
self
.derivatives.append(d)
def
forward_propagate(
self
, inputs):
activations = inputs
self
.activations[
0
] = inputs
for
i, w in
enumerate
(
self
.weights):
#calc the net inputs
net_inputs = np.dot(activations, w)
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 23
#calc the activations
activations = self
._sigmoid(net_inputs)
self
.activations[i
+
1
] = activations
return
activations
def
_sigmoid(
self
, x):
return
1 / (
1
+
np.exp(
-
x))
def
back_prop(
self
, error, verbose
=
False
):
#dE/dW_i = (y - a_[i+1]) s'(h_[i+1])) a_i
#s'(h_[i+1]) = s(h_[i+1])(1- s(h_[i+1]))
#s(h_[i+1]) = a_[i+1]
#dE/dW_[i-1] = (y - a_[i+1]) s'(h_[i+1])) W_i s'(h_i) a_[i-1]
for
i in
reversed
(
range
(
len
(
self
.derivatives))):
activations = self
.activations[i
+
1
]
delta = error * self
._sigmoid_derivative(activations) # ndarray([0.1, 0.2])--> ndarray([[0.1 , 0.2]])
delta_reshaped = delta.reshape(delta.shape[
0
], -
1
).T
current_activations = self
.activations[i] # ndarray([0.1, 0.2])--
> ndarray([[0.1], [0.2]])
current_activations_reshaped = current_activations.reshape(current_activations.shape[
0
],
-
1
)
self
.derivatives[i] = np.dot(current_activations_reshaped, delta_reshaped)
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 24
error = np.dot(delta, self
.weights[i].T)
if
verbose:
print
(
"Derivatives for W{}: {}"
.
format
(i, self
.derivatives[i]))
return
error
def
gradient_desceent(
self
, learning_rate):
for
i in
range
(
len
(
self
.weights)):
weights = self
.weights[i]
#print("Original W{} {}".format(i, weights))
derivatives = self
.derivatives[i]
weights += derivatives * learning_rate
#print("Updated W{} {}".format(i, weights))
def
train(
self
, inputs, targets, epochs, learning_rate):
for
i in
range
(epochs):
sum_error = 0
for
input
, target in
zip
(inputs, targets):
#pertform forward prop
output = self
.forward_propagate(
input
)
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 25
#calc the Error
error = target - output
#back prop
self
.back_prop(error, verbose
=
False
)
#APPLY gradient_descent
self
.gradient_desceent(learning_rate)
sum_error += self
._mse(target, output)
# report the error
print
(
"Error: {} at Epoch {}"
.
format
(sum_error / len
(inputs), i))
def
_mse(
self
, target, output):
return
np.average((target - output) ** 2
)
def
_sigmoid_derivative(
self
, x):
return
x * (
1.0 - x)
if
__name__ == "__main__"
:
# create a Multilayer Perceptron with one hidden layer
mlp = MLP(
2
, [
10
, 3
], 2
)
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 26
# train network
mlp.train(train_inputs, train_outputs, 100
, 0.1
) # get a prediction
hit = 0
;
for
i in
range
(np.shape(test_inputs)[
0
]):
y =
test_outputs[
0
]
output = mlp.forward_propagate(test_inputs[i])
if
(output[
0
]
>
output[
1
]):
if
(y[
0
]
==
1
):
hit
+=
1
else
:
if
(y[
1
]
==
1
):
hit
+=
1
print
(hit
/
np.shape(test_inputs)[
0
]) Appendix D: Tensorflow Keras MLP
import
tensorflow as
tf
from
sklearn.model_selection import
train_test_split
import
numpy as
np
from
random import
random
# create a dataset with 2000 samples
x_train
= train_inputs
x_test = test_inputs
y_train
= train_outputs
y_test = test_outputs
# build model with 3 layers: 2 -> 5 -> 1
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 27
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(
10
, input_dim
=
2
, activation
=
"relu"
), #relu, tanh, sigmoid or tf.keras.layers.LeakyReLU(alpha=0.01)
tf.keras.layers.Dense(
10
, activation
=
"relu"
),
tf.keras.layers.Dense(
1
, activation
=
"relu"
)
])
tf.keras.layers.Lea
# choose optimiser
optimizer = tf.keras.optimizers.SGD(learning_rate
=
0.087
)
# compile model
model.
compile
(optimizer
=
optimizer, loss
=
'mse'
)
# train model
model.fit(x_train, y_train, epochs
=
100
, batch_size=
5
)
# evaluate model on test set
print
(
"\nEvaluation on the test set:"
)
model.evaluate(x_test, y_test, verbose
=
2
) hit = 0
; miss = 0 missC
=
[] hitC
=
[] output = model.predict(x_test) for
i in
range
(np.shape(test_inputs)[
0
]): if
(output[i]
>
0.5
): output[i]
=
1 else
: output[i]
=
0 for
i in
range
(np.shape(test_inputs)[
0
]): if
(y_test[i]
==
output[i]): Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
MLP ASSIGNMENT, BY 28
hit
+=
1 hitC.append(test_inputs[i]) else
: miss
+=
1 missC.append(test_inputs[i]) print
(hit,miss) print
(hit
/
np.shape(test_inputs)[
0
]) hitC
=
np.array(hitC) missC
=
np.array(missC) plt.scatter(hitC[:,
0
], hitC[:,
1
], color
=
[
1
,
0
,
0
], s
=
s) plt.scatter(missC[:,
0
], missC[:,
1
], color
=
[
0
,
0
,
0.5
], s
=
5
) plt.show() Appendix E: confusion matrix generation
from
sklearn.metrics import
ConfusionMatrixDisplay
from
sklearn.metrics import
confusion_matrix
import
sklearn.metrics as
metrics
labels = [
"C1"
, "C2"
]
cm = confusion_matrix(y_test, output)
disp = ConfusionMatrixDisplay(confusion_matrix
=
cm, display_labels
=
labels)
disp.plot(cmap
=
plt.cm.Blues)
plt.show()
Downloaded by Lightening Azarakhsh (lightening.azarakhsh@gmail.com)
lOMoARcPSD|37606529
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help
Related Documents
Related Questions
No paper
arrow_forward
I need the expert solution and a detailed explanation of each step in the solution.
arrow_forward
De fine time constant (I) for (R-L) cct.
arrow_forward
HW1: Drive the Moore model state diagrams for a sequence 101 detector
and then check it for possible state reduction.
Hint: draw the state diagram (Moore model), convert the state diagram to
state table first and then use the implication chart to find the reduced state
table.
mra
arrow_forward
What theoretical transfer function models happen with a theoretical hvac system heating and cooling a home
arrow_forward
Please review the images.
arrow_forward
(B): Design a Moore FSM to detect the sequence (00) using T-FFs.
Q4/
(A): Design a Mealy FSM for odd parity generator using JK-FFs.
arrow_forward
Write a PIC16F877A program that flash ON the 8-LED's connected to port-B by using
two switches connected to port-D (Do & D₁) as shown in figure below, according to the
following scenarios: (Hint: Use 500ms delay for each case with 4MHz frequency)
1. When Do=1 then B₁,B3,B7 are ON.
2. When Do 0 then Bo,B2, B4, B5, B6 are ON.
3. When D=1 then B4, B5, B6, B7 are ON.
4. When D₁=0 then B0,B1,B2,B3 are ON.
U1
OSC1/CLION
33
OSC2/CLKOUT T 34
5
U2
20
10
35
3
18
10
- 30 -ou-un
RB2
2
RA1/AN1
PA3/AN3VREF+
RA4/TOCKIC1OUT
RACIANO 0RB3PGM 37
33
4
17
5
10
RB4
39
RA2/ANZ/VREF-CVREF
RB5
15
39
7
14
RB6PGC
40
13
RB7/PGO
RAS/AN4/SS/C2OUT
+
15
RCOT1OSOTICK
10
RECANARD
RE1/AN/R
REZ/ANTICS
✓ MCLR/Vpp/THV
RC2/CCP1
RC3/SCK/SCL
RCA/SOISDA
RC5/S00
RCB/TX/CK
10
RC1/T10SUCCP2
17
LED-BARGRAPH-RED
RC7/RX/DT
ROOPSPO
RD1/PSP1
R02/PSP2
RD3/PSP3
ROUPSP4
RD5PSP5
ROOPSPO
RO7/PSP7
PIC16F877A
+5V
R2
100R
R1
100R
arrow_forward
0.4/ For each pair of second-order system specifications that follow, find the
location of the second-order pair of poles.
a. %OS = 12%; Ts = 0.6 second
b. %OS = 10%; Tp
%3D
= 5 seconds
%3D
c. Ts =7 seconds; Tp=3 seconds
arrow_forward
For the system shown:
a. Get the equations
b. Simplify and determine C/R
c. Draw the signal flow diagram for this
system.
H3
Z3
m1
R
E
E1
G1
G2
G3
G4
m n
H2
Z1
22
H1
arrow_forward
SEE MORE QUESTIONS
Recommended textbooks for you

Introductory Circuit Analysis (13th Edition)
Electrical Engineering
ISBN:9780133923605
Author:Robert L. Boylestad
Publisher:PEARSON

Delmar's Standard Textbook Of Electricity
Electrical Engineering
ISBN:9781337900348
Author:Stephen L. Herman
Publisher:Cengage Learning

Programmable Logic Controllers
Electrical Engineering
ISBN:9780073373843
Author:Frank D. Petruzella
Publisher:McGraw-Hill Education

Fundamentals of Electric Circuits
Electrical Engineering
ISBN:9780078028229
Author:Charles K Alexander, Matthew Sadiku
Publisher:McGraw-Hill Education

Electric Circuits. (11th Edition)
Electrical Engineering
ISBN:9780134746968
Author:James W. Nilsson, Susan Riedel
Publisher:PEARSON

Engineering Electromagnetics
Electrical Engineering
ISBN:9780078028151
Author:Hayt, William H. (william Hart), Jr, BUCK, John A.
Publisher:Mcgraw-hill Education,
Related Questions
- HW1: Drive the Moore model state diagrams for a sequence 101 detector and then check it for possible state reduction. Hint: draw the state diagram (Moore model), convert the state diagram to state table first and then use the implication chart to find the reduced state table. mraarrow_forwardWhat theoretical transfer function models happen with a theoretical hvac system heating and cooling a homearrow_forwardPlease review the images.arrow_forward
- (B): Design a Moore FSM to detect the sequence (00) using T-FFs. Q4/ (A): Design a Mealy FSM for odd parity generator using JK-FFs.arrow_forwardWrite a PIC16F877A program that flash ON the 8-LED's connected to port-B by using two switches connected to port-D (Do & D₁) as shown in figure below, according to the following scenarios: (Hint: Use 500ms delay for each case with 4MHz frequency) 1. When Do=1 then B₁,B3,B7 are ON. 2. When Do 0 then Bo,B2, B4, B5, B6 are ON. 3. When D=1 then B4, B5, B6, B7 are ON. 4. When D₁=0 then B0,B1,B2,B3 are ON. U1 OSC1/CLION 33 OSC2/CLKOUT T 34 5 U2 20 10 35 3 18 10 - 30 -ou-un RB2 2 RA1/AN1 PA3/AN3VREF+ RA4/TOCKIC1OUT RACIANO 0RB3PGM 37 33 4 17 5 10 RB4 39 RA2/ANZ/VREF-CVREF RB5 15 39 7 14 RB6PGC 40 13 RB7/PGO RAS/AN4/SS/C2OUT + 15 RCOT1OSOTICK 10 RECANARD RE1/AN/R REZ/ANTICS ✓ MCLR/Vpp/THV RC2/CCP1 RC3/SCK/SCL RCA/SOISDA RC5/S00 RCB/TX/CK 10 RC1/T10SUCCP2 17 LED-BARGRAPH-RED RC7/RX/DT ROOPSPO RD1/PSP1 R02/PSP2 RD3/PSP3 ROUPSP4 RD5PSP5 ROOPSPO RO7/PSP7 PIC16F877A +5V R2 100R R1 100Rarrow_forward0.4/ For each pair of second-order system specifications that follow, find the location of the second-order pair of poles. a. %OS = 12%; Ts = 0.6 second b. %OS = 10%; Tp %3D = 5 seconds %3D c. Ts =7 seconds; Tp=3 secondsarrow_forward
arrow_back_ios
arrow_forward_ios
Recommended textbooks for you
- Introductory Circuit Analysis (13th Edition)Electrical EngineeringISBN:9780133923605Author:Robert L. BoylestadPublisher:PEARSONDelmar's Standard Textbook Of ElectricityElectrical EngineeringISBN:9781337900348Author:Stephen L. HermanPublisher:Cengage LearningProgrammable Logic ControllersElectrical EngineeringISBN:9780073373843Author:Frank D. PetruzellaPublisher:McGraw-Hill Education
- Fundamentals of Electric CircuitsElectrical EngineeringISBN:9780078028229Author:Charles K Alexander, Matthew SadikuPublisher:McGraw-Hill EducationElectric Circuits. (11th Edition)Electrical EngineeringISBN:9780134746968Author:James W. Nilsson, Susan RiedelPublisher:PEARSONEngineering ElectromagneticsElectrical EngineeringISBN:9780078028151Author:Hayt, William H. (william Hart), Jr, BUCK, John A.Publisher:Mcgraw-hill Education,

Introductory Circuit Analysis (13th Edition)
Electrical Engineering
ISBN:9780133923605
Author:Robert L. Boylestad
Publisher:PEARSON

Delmar's Standard Textbook Of Electricity
Electrical Engineering
ISBN:9781337900348
Author:Stephen L. Herman
Publisher:Cengage Learning

Programmable Logic Controllers
Electrical Engineering
ISBN:9780073373843
Author:Frank D. Petruzella
Publisher:McGraw-Hill Education

Fundamentals of Electric Circuits
Electrical Engineering
ISBN:9780078028229
Author:Charles K Alexander, Matthew Sadiku
Publisher:McGraw-Hill Education

Electric Circuits. (11th Edition)
Electrical Engineering
ISBN:9780134746968
Author:James W. Nilsson, Susan Riedel
Publisher:PEARSON

Engineering Electromagnetics
Electrical Engineering
ISBN:9780078028151
Author:Hayt, William H. (william Hart), Jr, BUCK, John A.
Publisher:Mcgraw-hill Education,