Thanks in advance.Subject Artificial Intellegent. Please provide me the code to do the following tasks by using (twitter_sexism_parsed_dataset.csv) data setTokenizeCleanVectorize usingCountVectorizationTF-IDFCreate new features (feature engineering)Length of text fieldPercentage of characters that are punctuation in the textUsing RandomForest, classify text into sexism or noneComplete the table below ModelPrecisionRecallAccuracy1CountVectorizer + Length 2CountVectorizer + Length + %punct 3CountVectorizer + Length + %punct 4TF-IDF + Length 5TF-IDF + Length + %punct 6TF-IDF + Length + %punct Show all graphsWrite one page description of your methodology.

Q. Subject Artificial Intellegent. Please provide me the code to do the following tasks by using (twitter_sexism_parsed_dataset.csv) data set Tokenize Clean Vectorize using CountVectorization TF-IDF Create new features (feature engineering) Length of text field Percentage of characters that are punctuation in the text Using RandomForest, classify text into sexism or none Complete the table belowAns. 1. Number of CharactersCount the quantity of characters present in a tweet. def count_chars(text): return len(text)2. Number of wordsCount the quantity of words present in a tweet. def count_words(text): return len(text.split())3. Number of capital charactersCount the quantity of capital characters present in a tweet. def count_capital_chars(text): count=0 for I in text: if i.isupper(): count+=1 bring count back 4. Number of capital wordsCount the quantity of capital words present in a tweet. def count_capital_words(text): return sum(map(str.isupper,text.split())) 5. Count the quantity of accentuationsIn this capacity, we return a word reference of 32 accentuation with the counts, which can be utilized as isolated highlights, which I will examine in the following area. def count_punctuations(text): punctuations='!"#$%&'()*+,- ./:;<=>?@[\]^_'{|}~' d=dict() for I in accentuations: d[str(i)+' count']=text.count(i) bring d back 6. Number of words in statementsThe quantity of words in the single citation and twofold citation. def count_words_in_quotes(text): x = re.findall("'.'|"."", text) count=0 on the off chance that x is None: bring 0 back else: for I in x: t=i[1:- 1] count+=count_words(t) bring count back 7. Number of sentencesInclude the quantity of sentences in a tweet. def count_sent(text): return len(nltk.sent_tokenize(text)) 8. Count the quantity of remarkable wordsCount the quantity of remarkable words in a tweet. def count_unique_words(text): return len(set(text.split()))9. Count of hashtagsSince we are utilizing the Twitter dataset we can count the occasions clients utilized the hashtag. def count_htags(text): x = re.findall(r'(#w[A-Za-z0-9]*)', text) return len(x)10. Count of noticesOn Twitter, more often than not individuals answer or notice somebody in their tweet, counting the quantity of notices can likewise be treated as a component. def count_mentions(text): x = re.findall(r'(@w[A-Za-z0-9]*)', text) return len(x)11. Count of stopwordsHere we will count the quantity of stopwords utilized in a tweet. def count_stopwords(text): stop_words = set(stopwords.words('english')) word_tokens = word_tokenize(text) stopwords_x = [w for w in word_tokens if w in stop_words] return len(stopwords_x)12. Computing normal word lengthThis can be determined by partitioning the counts of characters by counts of words. df['avg_wordlength'] = df['char_count']/df['word_count']13. Computing normal sentence lengthThis can be determined by partitioning the counts of words by the counts of sentences. df['avg_sentlength'] = df['word_count']/df['sent_count']14. novel words versus word count includeThis element is essentially the proportion of remarkable words to an all out number of words. df['unique_vs_words'] = df['unique_word_count']/df['word_count']15. Stopwords count versus words counts highlightThis element is additionally the proportion of counts of stopwords to the all out number of words. df['stopwords_vs_words'] = df['stopword_count']/df['word_count']

Answered: taset.csv) data set Tokenize

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

	Model	Precision	Recall	Accuracy
1	CountVectorizer + Length
2	CountVectorizer + Length + %punct
3	CountVectorizer + Length + %punct
4	TF-IDF + Length
5	TF-IDF + Length + %punct
6	TF-IDF + Length + %punct