Answered: Compute word frequencies for each word…

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Related questions

Question

# Pytorch Deep Learning(python)

# Answer according following code

import numpy as np

import json

img_codes = np.load("data/image_codes.npy")

captions = json.load(open('data/captions_tokenized.json'))

for img_i in range(len(captions)):

for caption_i inrange(len(captions[img_i])):

sentence = captions[img_i][caption_i]

captions[img_i][caption_i] = ["#START#"] + sentence.split(' ') + ["#END#"]

# Build a Vocabulary

from collections import Counter

word_counts = Counter()

# Compute word frequencies for each word in captions. See code above for data structure

# YOUR CODE HERE

#Check your solution below and Testing condition:-

vocab = ['#UNK#', '#START#', '#END#', '#PAD#']

vocab += [k for k, v in word_counts.items() if v >= 5 if k not in vocab]

n_tokens = len(vocab)

assert 10000 <= n_tokens <= 10500

#for reference and more detail go to ---> https://colab.research.google.com/github/hse-aml/intro-to-dl-pytorch/blob/main/week06/week06_final_project_image_captioning.ipynb#scrollTo=x_wH1bYu-QK8

Expert Solution

Trending now

This is a popular solution!

Step by step

Solved in 2 steps

SEE SOLUTION Check out a sample Q&A here

Knowledge Booster

Learn more about

Need a deep-dive on the concept behind this application? Look no further. Learn more about this topic, computer-science and related others by exploring similar questions and additional content below.

Similar questions