TB#3

Chapter 16, Deep Learning 1 Deep Learning 1 Introduction 16.1 Q1: Which of the following statements is false ? a. Keras offers a friendly interface to Google’s TensorFlow—the most widely used deep-learning library. b. François Chollet of the Google Mind team developed Keras to make deep- learning capabilities more accessible. c. Keras enables you to define deep-learning models conveniently with one statement. d. Google has thousands of TensorFlow and Keras projects underway internally, and that number is growing quickly. Answer: c. Keras enables you to define deep-learning models conveniently with one statement. Actually, deep learning models require more sophisticated setups than scikit-learn machine learning models, typically using several statements to connect multiple objects, called layers. 16.1 Q2: Which of the following statements is false ? a. Keras is to deep learning as Scikit-learn is to machine learning. b. Deep learning models are complex and require an extensive mathematical background to understand their inner workings. c. Both Keras and Scikit-learn encapsulate the sophisticated mathematics of their models, so developers simply need only define, parameterize and manipulate objects. d. With Keras, you build your models primarily from custom components you develop to meet your unique requirements. Answer: d. With Keras, you build your models primarily from custom components you develop to meet your unique requirements. Actually, with Keras, you build your models from pre-existing components and quickly parameterize those components to your unique requirements. 16.1 Q3: Which of the following statements is false ? a. Keras facilitates experimenting with many deep-learning models and tweaking them in various ways until you find the models that perform best for your applications. b. Deep learning works well only when you have lots of data. c. Transfer learning uses existing knowledge from a previously trained model as the foundation for a new model. d. Data augmentation adds data to a dataset by deriving new data from existing data. For example, in an image dataset, you might rotate the images left and right so the model can learn about objects in different orientations.

2 Chapter 16, Deep Learning Answer: b. Deep learning works well only when you have lots of data. Actually, deep learning works well when you have lots of data, but it also can be effective for smaller datasets, especially when combined with techniques like transfer learning and data augmentation. 16.1 Q4: Which of the following statements a), b) or c) is false? a. Deep learning can require significant processing power. b. Complex models trained on big-data datasets can take hours, days or even more to train. c. Special high-performance hardware called GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units) developed by NVIDIA and Google, respectively, to meet the extraordinary processing demands of edge-of-the- practice- deep-learning applications. d. All of the above statements are true . Answer: d. All of the above statements are true . 16.1 Q5: ________ neural networks are especially appropriate for computer vision tasks, such as recognizing handwritten digits and characters or recognizing objects (including faces) in images and videos. a. LSTM b. Recurrent c. Convolutional d. None of the above Answer: c. Convolutional 16.1 Q6: Which of the following are not automated deep-learning capabilities? a. Auto-Keras from Texas A&M University’s DATA Lab b. Baidu’s EZDL c. Google’s AutoML d. Scikit-learn Answer: d. Scikit-learn 1 Deep Learning Applications 16.1 Q7: Which of the following are popular deep learning applications: a. Game playing, computer vision, self-driving cars, robotics, improving customer experiences and chatbots b. Diagnosing medical conditions, Google Search, facial recognition, automated image captioning, video closed captioning, enhancing image resolution c. Speech recognition, language translation, predicting election results, predicting earthquakes and weather. d. All of the above Answer: All of the above

4 Chapter 16, Deep Learning with Anaconda are installed into the root environment and, unless you specify otherwise, any additional libraries you install also are placed there. d. Custom environments give you control over the specific libraries you wish to install for your specific tasks. Answer: c. The default environment in Anaconda is called the root environment. This is created for you when you install Anaconda. All the Python libraries that come with Anaconda are installed into the root environment and, unless you specify otherwise, any additional libraries you install also are placed there. Actually, the default environment in Anaconda is called the base environment . 16.3 Q2: Which of the following statements a), b) or c) is false? a. To use a custom Anaconda environment named tf_env , execute the following command, which affects only the current Terminal, shell or Anaconda Command Prompt: conda activate tf_env b. When a custom environment is activated and you install more libraries, they become part of the activated environment, not the base environment. c. If you open separate Terminals, shells or Anaconda Command Prompts, they’ll use Anaconda’s base environment by default. d. All of the above statements are true . Answer: d. All of the above statements are true . 1 Neural Networks 16.4 Q1: Which of the following statements a), b) or c) is false? a. Deep learning is a form of machine learning that uses artificial neural networks to learn. b. An artificial neural network is a software construct that operates similarly to how scientists believe our brains work. c. Our biological nervous systems are controlled via neurons that communicate with one another along pathways called synapses . As we learn, the specific neurons that enable us to perform a given task, like walking, communicate with one another more efficiently. d. All of the above statements are true . Answer: d. All of the above statements are true . 16.4 Q2: In supervised deep learning, we aim to predict the ________ labels supplied with data samples. a. goal b. mark c. object d. target

Chapter 16, Deep Learning 5 Answer: d. target 16.4 Q3: The following diagram shows a three- layer neural network. Each circle represents a neuron, and the lines between them simulate the synapses. The output of a neuron becomes the input of another neuron, hence the term neural network: This particular diagram shows a ________. a. partially connected network b. crossover network c. fully connected network d. None of the above Answer: c. fully connected network 16.4 Q4: Which of the following statements a), b) or c) about neural networks is false ? a. During the training phase, the network calculates values called weights for every connection between the neurons in one layer and those in the next. b. On a neuron-by-neuron basis, each of its inputs is multiplied by that connection’s weight, then the maximum of those weighted inputs is passed to the neuron’s activation function. c. The activation function’s output determines which neurons to activate based on the inputs—just like the neurons in your brain passing information around in response to inputs coming from your eyes, nose, ears and more. d. All of the above statements are true . Answer: b. On a neuron-by-neuron basis, each of its inputs is multiplied by that connection’s weight, then the maximum of those weighted inputs is passed to the neuron’s activation function. Actually, on a neuron-by-neuron basis, each of its inputs is multiplied by that connection’s weight, then the sum of those weighted inputs is passed to the neuron’s activation function. 16.4 Q5: Which of the following statements about the diagram below is false ?

Chapter 16, Deep Learning 7 d. These tensors can become quite large as the number of dimensions increases and as the richness of the data increases (for example, images, audios and videos are richer than text). Answer: b. A tensor is basically a one-dimensional array. Actually, A tensor is basically a multidimensional array . 16.5 Q2: Chollet discusses the types of tensors typically encountered in deep learning: A 0D (0-dimensional) tensor is one value and is known as a scalar. A 1D tensor is similar to a one-dimensional array and is known as a vector. A 1D tensor might represent a sequence, such as hourly temperature readings from a sensor or the words of one movie review. A 2D tensor is similar to a two- dimensional array and is known as a matrix. A 2D tensor could represent a grayscale image in which the tensor’s two dimensions are the image’s width and height in pixels, and the value in each element is the intensity of that pixel. Which of the following statements a), b) or c) about additional types of tensors is false ? a. A 3D tensor is similar to a three-dimensional array and could be used to represent a color image. The first two dimensions would represent the width and height of the image in pixels and the depth at each location might represent the red, green and blue (RGB) components of a given pixel’s color. A 3D tensor also could represent a collection of 2D tensors containing grayscale images. b. A 4D tensor could be used to represent a collection of color images in 3D tensors. It also could be used to represent one video. Each frame in a video is essentially a color image. c. A 5D tensor could be used to represent a collection of 4D tensors containing videos. d. All of the above statements are true . Answer: d. All of the above statements are true . 16.5 Q3: A tensor’s ________ typically is represented in Python as a tuple of values in which the number of elements specifies the tensor’s number of dimensions and each value in the tuple specifies the size of the tensor’s corresponding dimension. a. shape b. frame c. pattern d. format Answer: a. shape. 16.5 Q4: Which of the following statements a), b) or c) is false ? a. Powerful processors are needed for real-world deep learning because the size of tensors can be enormous and large-tensor operations can place crushing demands on processors.

8 Chapter 16, Deep Learning b. NVIDIA GPUs (Graphics Processing Units)—originally developed for computer gaming—are optimized for the mathematical matrix operations typically performed on tensors, an essential aspect of how deep learning works “under the hood.” c. Recognizing that deep learning is crucial to its future, Google developed TPUs (Tensor Processing Units). Google now uses TPUs in its Cloud TPU service, which can perform quadrillions of floating-point operations per second. d. All of the above statements are true . Answer: d. All of the above statements are true . 1 Convolutional Neural Networks for Vision; Multi-Classification with the MNIST Dataset 16.6 Q1: Which of the following statements a), b) or c) is false ? a. In the “Machine Learning” chapter, we classified handwritten digits using the 8-by-8-pixel, low-resolution images from the Digits dataset bundled with Scikit- learn. b. The Digits dataset is based on a subset of the higher-resolution MNIST handwritten digits dataset. c. Recurrent neural networks are common in computer-vision applications, such as recognizing handwritten digits and characters, and recognizing objects in images and video. d. All of the above statements are true . Answer: c. Recurrent neural networks are common in computer-vision applications, such as recognizing handwritten digits and characters, and recognizing objects in images and video. Actually, convolutional neural networks are common in computer-vision applications, such as recognizing handwritten digits and characters, and recognizing objects in images and video. 16.6 Q2: Which of the following statements a), b) or c) is false ? a. Reproducibility is crucial in scientific studies. b. In deep learning, reproducibility is more difficult because the libraries sequentialize operations that perform floating-point calculations. c. Getting reproducible results in Keras requires a combination of environment settings and code settings that are described in the Keras FAQ. d. All of the above statements are true . Answer: b. In deep learning, reproducibility is more difficult because the libraries sequentialize operations that perform floating-point calculations. Actually, in deep learning, reproducibility is more difficult because the libraries heavily parallelize operations that perform floating-point

10 Chapter 16, Deep Learning When you call load_data it will download the MNIST data to your system. The function returns a tuple of two elements containing the training and testing sets. Each element is itself a tuple containing the samples and labels, respectively. Answer: c. TensorFlow uses Keras to execute the deep-learning models. Actually, Keras uses TensorFlow to execute the deep-learning models. 1 Data Exploration 16.6 Q6: Which of the following statements a), b) or c) is false? a. You should always get to know the data before working with it. b. The following snippets check the dimensions of the MNIST training set images ( X_train ), training set labels ( y_train ), testing set images ( X_test ) and testing set labels ( y_test ): [3]: X_train.shape [3]: (60000, 28, 28) [4]: y_train.shape [4]: (60000,) [5]: X_test.shape [5]: (10000, 28, 28) [6]: y_test.shape [6]: (10000,) c. You can see from X_train ’s and X_test ’s shapes that the MNIST images are the same resolution as those in Scikit-learn’s Digits dataset. d. All of the above statements are true . Answer: c. You can see from X_train ’s and X_test ’s shapes that the MNIST images are the same resolution as those in Scikit-learn’s Digits dataset. Actually, you can see from X_train ’s and X_test ’s shapes that the images are higher resolution than those in Scikit-learn’s Digits dataset (which are 8-by-8). 16.6 Q7: The IPython magic ________ indicates that Matplotlib-based graphics should be displayed in a Jupyter notebook rather than in separate windows. a. %matplotlib notebook b. %matplotlib inline c. %matplotlib Jupyter d. None of the above Answer: b. %matplotlib inline

Chapter 16, Deep Learning 11 16.6 Q8: Consider the following code: import numpy as np index = np.random.choice(np.arange(len(X_train)), 24 , replace= False )

2 Chapter 16, Deep Learning b. For MNIST, each image’s width and height are 28 pixels, and each pixel has one channel (the grayscale shade of the pixel from 0 to 255), so each sample’s shape will be: (28, 28, 1) c. Full-color images with RGB (red/green/blue) values for each pixel, would have three channels —one channel each for the red, green and blue components of a color. d. As the neural network learns from the images, it reduces the number of channels. Answer: d. As the neural network learns from the images, it reduces the number of channels. Actually, as the neural network learns from the images, it creates many more channels . 16.6 Q11: Which of the following statements a), b) or c) is false ? a. Numeric features in data samples may have value ranges that vary widely. Deep learning networks perform better on data that is scaled either into the range 0.0 to 1.0, or to a range for which the data’s mean is 1.0 and its standard deviation is 0.0. Getting your data into one of these forms is known as normalization. b. In MNIST, each pixel is an integer in the range 0–255. c. The following code converts the values to 32-bit (4-byte) floating-point numbers using the NumPy array method astype , then divides every element in the resulting array by 255, producing normalized values in the range 0.0–1.0: [16]: X_train = X_train.astype( 'float32' ) / 255 [17]: X_test = X_test.astype( 'float32' ) / 255 d. All of the above statements are true. Answer: a. Numeric features in data samples may have value ranges that vary widely. Deep learning networks perform better on data that is scaled either into the range 0.0 to 1.0, or to a range for which the data’s mean is 1.0 and its standard deviation is 0.0. Getting your data into one of these forms is known as normalization. Actually, deep learning networks perform better on data that is scaled either into the range 0.0 to 1.0, or to a range for which the data’s mean is 0.0 and its standard deviation is 1.0 . 16.6 Q12: The MNIST convnet’s prediction for each MNIST digit will be an array of 10 probabilities, indicating the likelihood that the digit belongs to a particular one of the classes 0 through 9. When we evaluate the model’s accuracy, Keras compares the model’s predictions to the labels. To do that, Keras requires both to have the same ________. a. profile

Chapter 16, Deep Learning 3 b. aspect c. shape d. frame Answer: c. shape 16.6 Q13: Which of the following statements a), b) or c) is false? a. One-hot encoding, which converts data into arrays of 1.0s and 0.0s in which only one element is 1.0 and the rest are 0.0s. b. For MNIST targets, the one-hot-encoded values will be 10 x 10 arrays representing the categories 0 through 9. c. We know precisely which category each digit belongs to, so the categorical representation of a digit label will consist of a 1.0 at that digit’s index and 0.0s for all the other elements. d. All of the above statements are true . Answer: b. For MNIST targets, the one-hot-encoded values will be 10 x 10 arrays representing the categories 0 through 9. Actually, the one-hot encoded values will be one-dimensional 10-element arrays. 16.6 Q14: Which of the following statements a), b) or c) is false ? a. The one-hot encoded representation of the digit 7 is: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0] b. The tensorflow.keras.utils module provides function to_categorical to perform one-hot encoding. c. The function counts the unique categories then, for each item being encoded, creates an array of that length with a 1.0 in the correct position. d. All of the above statements are true . Answer: a. The one-hot encoded representation of the digit 7 is: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0] Actually, the one-hot encoded representation of the digit 7 is: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0] . 1 Creating the Neural Network 16.6 Q15: Which of the following statements a), b) or c) is false ? a. The following code begins configuring a convolutional neural network with a Keras Sequential model from the tensorflow.keras.models module: [24]: from tensorflow.keras.models import Sequential [25]: cnn = Sequential() b. The network resulting from the code in Part (a) will execute its layers sequentially—the output of one layer becomes the input to the next.

Chapter 16, Deep Learning 5 Which of the following statements a), b) or c) is false ? a. The areas that convolution learns from are called kernels or patches. b. When the kernel reaches the right edge, the convolution layer moves the kernel down three pixels and repeats this left-to-right process. c. Kernels typically are 3-by-3, though larger convnets can be used for higher- resolution images. d. All of the above statements are true . Answer: b. When the kernel reaches the right edge, the convolution layer moves the kernel down three pixels and repeats this left-to-right process. Actually, when the kernel reaches the right edge, the convolution layer moves the kernel one pixel down and repeats this left-to-right process. 16.6 Q19: Which of the following statements a), b) or c) about convolution is false ? a. Kernel-size is a tunable hyperparameter. b. For each kernel position, the convolution layer performs mathematical calculations using the kernel features to “learn” about them, then outputs one new feature to the layer’s output. c. By looking at features near one another, the network begins to recognize features like edges, straight lines and curves. d. All of the above statements are true . Answer: d. All of the above statements are true . Answer: d. All of the above statements are true . 16.6 Q20: Which of the following statements is false ? a. The number of filters depends on the image dimensions—higher-resolution images have more features, so they require more filters. b. If you study the code the Keras team used to produce their pretrained convnets, you’ll find that they used 64, 128 or even 256 filters in their first

6 Chapter 16, Deep Learning convolutional layers. Based on their convnets and the fact that the MNIST images are small, we used 64 filters in our first convolutional layer. c. The set of filters produced by a convolution layer is called a feature map. Subsequent convolution layers combine features from previous feature maps to recognize larger features and so on. If we were doing facial recognition, early layers might recognize lines, edges and curves, and subsequent layers might begin combining those into larger features like eyes, eyebrows, noses, ears and mouths. d. Once the network learns a feature, because of convolution, it no longer needs to recognize that feature elsewhere in the image. Answer: d. Once the network learns a feature, because of convolution, it no longer needs to recognize that feature elsewhere in the image. Actually, Once the network learns a feature, because of convolution, it can recognize that feature anywhere in the image—this is one of the reasons that convnets are used for object recognition in images. 16.6 Q21: Which of the following statements is false ? a. The following code adds a Conv2D convolution layer to a model named cnn : cnn.add(Conv2D(filters= 64 , kernel_size=( 3 , 3 ), activation= 'relu' , input_shape=( 28 , 28 , 1 ))) b. The Conv2D layer in Part (a) is configured with the following arguments: • filters= 64 —The number of filters in the resulting feature map. • kernel_size=( 3 , 3 ) —The size of the kernel used in each filter. • activation= 'relu' —The 'relu' (Rectified Linear Unit) activation function is used to produce this layer’s output. 'relu' is the most widely used activation function in today’s deep learning networks and is good for performance because it’s easy to calculate. It’s commonly recommended for convolutional layers. c. Assuming the Conv2D layer in Part (a) is the first layer of the network, we also pass the input_shape=(28, 28,1) argument to specify the shape of each sample. This automatically creates an input layer to load the samples and pass them into the Conv2D layer, which is actually the first hidden layer . d. In Keras, for each subsequent layer you must explicitly specify the layer’s input_shape to match the previous layer’s output shape, making it possible to stack layers. Answer: d. In Keras, for each subsequent layer you must explicitly specify the layer’s input_shape to match the previous layer’s output shape, making it possible to stack layers. Actually, in Keras, each subsequent layer

8 Chapter 16, Deep Learning Which of the following statements a), b) or c) is false ? a. The max pooling layer first looks at the pool in position 1 above, then outputs the maximum feature from that pool—9 in our diagram. b. Unlike convolution, there’s no overlap between pools. Once the pool reaches the right edge, the pooling layer moves the pool down by its height—2 rows— then continues from left-to-right. Because of the feature reduction in each group, 2-by-2 pooling compresses the number of features by 50%. c. The following code adds a MaxPooling2D layer to a model named cnn : cnn.add(MaxPooling2D(pool_size=( 2 , 2 ))) d. All of the above statements are true. Answer: b. Unlike convolution, there’s no overlap between pools. Once the pool reaches the right edge, the pooling layer moves the pool down by its height—2 rows—then continues from left-to-right. Because of the feature reduction in each group, 2-by-2 pooling compresses the number of features by 50%. Actually, because every group of four features is reduced to one, 2- by-2 pooling compresses the number of features by 75%. 16.6 Q25: Which of the following statements is false ? a. Convnets often have many convolution and pooling layers. b. The Keras team’s convnets tend to double the number of filters in subsequent convolutional layers to enable the model to learn more relationships between the features. c. The following snippets add a convolution layer with 128 filters, followed by a pooling layer to reduce the dimensionality by 50%: cnn.add(Conv2D(filters= 128 , kernel_size=( 3 , 3 ), activation= 'relu' ))

Chapter 16, Deep Learning 9 cnn.add(MaxPooling2D(pool_size=( 2 , 2 ))) d. For odd dimensions like 11-by-11, Keras pooling layers round down by default. Answer: c. The following snippets add a convolution layer with 128 filters, followed by a pooling layer to reduce the dimensionality by 50%: cnn.add(Conv2D(filters= 128 , kernel_size=( 3 , 3 ), activation= 'relu' )) cnn.add(MaxPooling2D(pool_size=( 2 , 2 ))) Actually, the pooling layer in the preceding code reduce the dimensionality of the Conv2D layer’s output by 75% : 16.6 Q26: A Keras ________ layer reshapes its input to one dimension. a. Dropout b. Masking c. Dense d. Flatten Answer: d. Flatten 16.6 Q27: Which of the following statements a), b) or c) is false? a. Learning the relationships among features and performing classification is accomplished with partially connected Dense layers. b. The following Dense layer creates 128 neurons ( units ) that learn from the outputs of the previous layer: cnn.add(Dense(units= 128 , activation= 'relu' )) c. Many convnets contain at least one Dense layer like the one above. Convnets geared to more complex image datasets with higher-resolution images like ImageNet ( http://www.image-net.org )—a dataset of over 14 million images— often have several Dense layers, commonly with 4096 neurons. d. All of the above statements are true. Answer: a. Learning the relationships among features and performing classification is accomplished with partially connected Dense layers. Actually, learning the relationships among features and performing classification is accomplished with fully connected Dense layers. 16.6 Q28: Consider the following code: cnn.add(Dense(units= 10 , activation= 'softmax' ))