Skip to content

Instantly share code, notes, and snippets.

@kshirsagarsiddharth

kshirsagarsiddharth/Stop.py

Last active December 27, 2022 10:30
Show Gist options
  • Save kshirsagarsiddharth/70698cb0de8dbf990d9bd7c80a8ed9fc to your computer and use it in GitHub Desktop.
Save kshirsagarsiddharth/70698cb0de8dbf990d9bd7c80a8ed9fc to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Stop.py
import nltk
from nltk.corpus import stopwords
# Load the stop words
stop_words = set(stopwords.words('english'))
def remove_stop_words(text):
# Tokenize the text
tokens = nltk.word_tokenize(text)
# Remove stop words
clean_tokens = [token for token in tokens if token not in stop_words]
# Join the clean tokens into a single string
clean_text = ' '.join(clean_tokens)
return clean_text
# Test the function
text = "This is some sample text with stop words."
remove_stop_words(text)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment