In this Python NLP lesson we are going to learn about NLP Frequency Distribution, using Frequency Distribution we can find that how many times a word is used in a sentence, or we can count words in our text using NLP Frequency Distribution.
Now let’s create an example, we are going to use some text from nltk.book and we want to find the frequency distribution of the words.
|
1 2 3 4 |
from nltk.book import text1 from nltk.probability import FreqDist freq_distrib = FreqDist(text1) freq_distrib |
If you run the code in Jupyter Notebook, this will be the result.
|
1 2 3 |
FreqDist({',': 18713, 'the': 13721, '.': 6862, 'of': 6536, 'and': 6024, 'a': 4569, 'to': 4542, ';': 4072, 'in': 3916, 'that': 2982, ...}) |
The class FreqDist works like a dictionary where the keys are the words in the text and the values are the count associated with that word. For example, if you want to see how many words “book” are in the text, you can type:
|
1 |
print(freq_distrib["Book"]) |
Now let’s create our own text and find the frequency distribution, you can see that we have a simple paragraph and we want find the frequency distribution of the text word, if you run the code you will receive 2 as the result, because we have two times text word.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
from nltk.tokenize import word_tokenize from nltk.probability import FreqDist text = "Hello friends, this our text," \ " this text is related to frequency distribution" text_list = word_tokenize(text) freqDist = FreqDist(text_list) words = list(freqDist.keys()) print(freqDist['text']) |
