Question 3 Inner Product of Vectors Before training, A and B are matrices with random values Ao and Bo. After training, we will obtain new weights A* and B. At each step of training (pro- gressively improving matrix weights through a process called gradient descent), vo = Y1 Y2 = = BoAou V₁ = = = B₁A₁u = BAu Y2 where u R³ is the input vector, vo E R² is the pre-trained output vector, and v is the trained output. Now let us check how close the output is to the expected classification. Let y₁ = ½, y₁ = 1, y = and y2 = ==. Confirm that these values are getting progressively closer to the target by computing (vo, √00, (b)), √(v1, (b)), and (v., (b)), ((8), (c) with (,): : R² × R² → R given by = ac+ bd This gives us a notion of length and "closeness" of vectors. We will study a special class of functions like (,) (that map two vectors to R), called inner products in the next few weeks. Question 4 Limitations of Linear Neural Networks a) Prove that the composition of linear transformations is linear. Conclude that the transformation given by the linear neural network in this problem NR3 R² is linear. b) Explain why, even though we have 20 links between 9 nodes, there are only 6 degrees of freedom in our linear neural network. This is why artificial neural networks rely on nonlinear "activation functions".

Algebra & Trigonometry with Analytic Geometry

13th Edition

ISBN:9781133382119

Author:Swokowski

Publisher:Swokowski

Chapter9: Systems Of Equations And Inequalities

Section9.8: Determinants

Problem 6E

See similar textbooks

Related questions

Question

Key Concepts / Background: we study how to train a neural network to classify data with 3 features into 2 classes. Through this example, we will become more familiar with the descriptive power of linear transformations and observe that purely linear neural networks (without nonlinear activation functions) are not good models for machine learning. If you want to ignore details about applications to machine learning, you can skip the underlined or blue text. For each

u ∈ R³in the training set, the desired output of the neural network for u is either

Traditionally, neural networks are comprised of a composition of linear transformations of data (neuron edge weights) and nonlinear transformations (activation functions), although we will not include activation functions here.