Suppose that when performing attention, we have the following keys and values:Keys:Values:{[−3 0 1],[1 1 −1],[0 1 1],[100]}{[211] [122], [031], [-1 0 2]}We want to compute the attention embedding using these keys and values for the following query:[2 1 -1]Which of the following is the correct attention embedding?To simplify calculations, replace softmax with argmax. For example, softmax([-1,1,0])would instead be argmax ([-1, 1, 0]) = [0, 1, 0].[2 1 1][122][031][-3 0 1]

Answered: Suppose that when performing attention,…

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Related questions

Concept explainers

Question

Suppose that when performing attention, we have the following keys and values:
Keys:
Values:
{[−3 0 1],[1 1 −1],[0 1 1],[100]}
{[211] [122], [031], [-1 0 2]}
We want to compute the attention embedding using these keys and values for the following query:
[2 1 -1]
Which of the following is the correct attention embedding?
To simplify calculations, replace softmax with argmax. For example, softmax([-1,1,0])
would instead be argmax ([-1, 1, 0]) = [0, 1, 0].
[2 1 1]
[122]
[031]
[-3 0 1]

Expert Solution

Step 1

Solution:

To compute the attention embedding, we need to first calculate the attention weights, which are computed as the dot product between the query and each key vector, followed by a softmax operation to normalize the results. Then, we compute the weighted sum of the value vectors using the attention weights as the weights.

Trending now

This is a popular solution!

Step by step

Solved in 2 steps

SEE SOLUTION Check out a sample Q&A here

Knowledge Booster

Learn more about

Need a deep-dive on the concept behind this application? Look no further. Learn more about this topic, computer-science and related others by exploring similar questions and additional content below.

Similar questions