Question 3 Inner Product of Vectors Before training, A and B are matrices with random values Ao and Bo. After training, we will obtain new weights A* and B. At each step of training (pro- gressively improving matrix weights through a process called gradient descent), vo = Y1 Y2 = = BoAou V₁ = = = B₁A₁u = BAu Y2 where u R³ is the input vector, vo E R² is the pre-trained output vector, and v is the trained output. Now let us check how close the output is to the expected classification. Let y₁ = ½, y₁ = 1, y = and y2 = ==. Confirm that these values are getting progressively closer to the target by computing (vo, √00, (b)), √(v1, (b)), and (v., (b)), ((8), (c) with (,): : R² × R² → R given by = ac+ bd This gives us a notion of length and "closeness" of vectors. We will study a special class of functions like (,) (that map two vectors to R), called inner products in the next few weeks. Question 4 Limitations of Linear Neural Networks a) Prove that the composition of linear transformations is linear. Conclude that the transformation given by the linear neural network in this problem NR3 R² is linear. b) Explain why, even though we have 20 links between 9 nodes, there are only 6 degrees of freedom in our linear neural network. This is why artificial neural networks rely on nonlinear "activation functions".

Linear Algebra: A Modern Introduction

4th Edition

ISBN:9781285463247

Author:David Poole

Publisher:David Poole

Chapter3: Matrices

Section3.1: Matrix Operations

Problem 20EQ: Referring to Exercise 19, suppose that the unit cost of distributing the products to stores is the...

See similar textbooks

Related questions

Question

Key Concepts / Background: we study how to train a neural network to classify data with 3 features into 2 classes. Through this example, we will become more familiar with the descriptive power of linear transformations and observe that purely linear neural networks (without nonlinear activation functions) are not good models for machine learning. If you want to ignore details about applications to machine learning, you can skip the underlined or blue text. For each

u ∈ R³in the training set, the desired output of the neural network for u is either

Traditionally, neural networks are comprised of a composition of linear transformations of data (neuron edge weights) and nonlinear transformations (activation functions), although we will not include activation functions here.