阻止过程在Python中不起作用

我有一个文本文件,在删除了停用词之后,我试图阻止它运行,但是运行该文件似乎没有任何变化.我的文件称为data0.

这是我的代码：

## Removing stopwords and tokenizing by words (split each word)
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

data0 = word_tokenize(data0)
data0 = ' '.join([word for word in data0 if word not in (stopwords.words('english'))])
print(data0)

## Stemming the data
from nltk.stem import PorterStemmer

ps = PorterStemmer()
data0 = ps.stem(data0)
print(data0)

我得到以下内容(为清晰起见而包装)：

For us around Aberdeen , question “ What oil industry ? ( Evening Express , October 26 ) touch deja vu . That question asked almost since day first drop oil pumped North Sea . In past 30 years seen constant cycle ups downs , booms busts industry . I predict happen next . There period worry uncertainty scrabble find something keep local economy buoyant oil gone . Then upturn see jobs investment oil , everyone breathe sigh relief quest diversify go back burner . That downfall . Major industries prone collapse . Look nation ‘s defunct shipyards extinct coal steel industries . That ‘s vital n’t panic downturns , start planning sensibly future . Our civic business leaders need constantly looking something secure prosperity – tourism , technology , bio-science emerging industries . We need economically strong rather waiting see happens oil roller coaster hits buffers . N JonesEllon

该代码的第一部分工作正常(删除停用词并标记化),但是第二部分(Stem)无效.知道为什么吗？

解决方法:

从Stemmer docs http://www.nltk.org/howto/stem.html看来,Stemmer旨在一次被单个单词调用.

尝试对每个单词运行

[word for word in data0 if word not in (stopwords.words('english'))]

即致电加入之前

例如.

stemmed_list = []
for str in [word for word in data0 if word not in (stopwords.words('english'))]:
    stemmed_list.append(ps.stem(str))

编辑：评论响应.
我运行了以下命令-它似乎可以正确阻止：

>>> from nltk.stem import PorterStemmer
>>> ps = PorterStemmer()
>>> data0 = '''<Your Data0 string>'''
>>> words = data0.split(" ")
>>> stemmed_words = map(ps.stem, words)
>>> print(list(stemmed_words))  # list cast needed because of 'map'
[..., 'industri', ..., 'diversifi']

我认为没有可以直接应用于文本的词干分析器,但是您可以将其包装在自己的接受“ ps”和文本的函数中：

def my_stem(text, stemmer):
    words = text.split(" ")
    stemmed_words = map(stemmer, words)
    result = " ".join(list(stemmed_words))
    return result

标签：stemming,python
来源： https://codeday.me/bug/20191027/1942460.html