使用NLTK語料庫進行荷蘭語鳴叫的情緒分析conll2002

Question

我需要對荷蘭語鳴叫列表進行情感分析，我正在使用conll2002。下面是我使用的代碼：

import nltk.classify.util 
from nltk.classify import NaiveBayesClassifier 
from nltk.corpus import conll2002 
import time 

t=time.time() 

def word_feats(words): 
    return dict([(word, True) for word in words]) 

#negids = conll2002.fileids('neg') 
def train(): 
    #negids = conll2002.fileids('neg') 
    #posids = conll2002.fileids('pos') 
    negids = conll2002.fileids() 
    posids = conll2002.fileids() 

    negfeats = [(word_feats(conll2002.words(fileids=[f])), 'neg') for f in negids] 
    posfeats = [(word_feats(conll2002.words(fileids=[f])), 'pos') for f in posids] 

    negcutoff = len(negfeats)*3/4 
    poscutoff = len(posfeats)*3/4 

    trainfeats = negfeats[:negcutoff] + posfeats[:poscutoff] 
    testfeats = negfeats[negcutoff:] + posfeats[poscutoff:] 
    print 'train on %d instances, test on %d instances' % (len(trainfeats), len(testfeats)) 

    classifier = NaiveBayesClassifier.train(trainfeats) 
    print 'accuracy:', nltk.classify.util.accuracy(classifier, testfeats) 
    classifier.show_most_informative_features() 
x=train() 
print x 
print time.time()-t 

上面的代碼工作，但輸出如下：

train on 8 instances, test on 4 instances 
accuracy: 0.5 
Most Informative Features 
       poderlas = True    pos : neg =  1.0 : 1.0 
        voert = True    pos : neg =  1.0 : 1.0 
      contundencia = True    pos : neg =  1.0 : 1.0 
      encuestocracia = None    pos : neg =  1.0 : 1.0 
       alivien = None    pos : neg =  1.0 : 1.0 
        Bogotá = True    pos : neg =  1.0 : 1.0 
      Especialidades = True    pos : neg =  1.0 : 1.0 
     hoofdredacteurs = True    pos : neg =  1.0 : 1.0 
       quisieron = True    pos : neg =  1.0 : 1.0 
       asciendan = None    pos : neg =  1.0 : 1.0 
None 
9.21083234 

POS機：NEG比出來是1：1的所有情況。我該如何解決這個問題？我想這個問題可能是住在我的代碼當中下面的語句：

negids = conll2002.fileids('neg') 
posids = conll2002.fileids('pos') 

如果我不註釋掉上述兩種說法，我得到的錯誤是：

Traceback (most recent call last): 
    File "naive1.py", line 31, in <module> 
    x=train() 
    File "naive1.py", line 13, in train 
    negids = conll2002.fileids('neg') 
TypeError: fileids() takes exactly 1 argument (2 given) 

我試圖用自己來解決這個問題，但它仍然無法正常工作。有人可以請指點我正確的方向嗎？提前致謝。

Answer 1

fileids()方法接受categories參數，但只在分類的語料庫中。例如：

>>> from nltk.corpus import brown 
>>> brown.fileids("mystery") 
['cl01', 'cl02', 'cl03', 'cl04', 'cl05', 'cl06', 'cl07', 'cl08', 'cl09', 
'cl10', 'cl11', 'cl12', 'cl13', 'cl14', 'cl15', 'cl16', 'cl17', 'cl18', 
'cl19', 'cl20', 'cl21', 'cl22', 'cl23', 'cl24'] 

由於CONLL語料庫沒有類別，您的呼叫失敗。這是因爲他們沒有註釋情緒：CONLL 2000和CONLL 2002分別是語料庫（分別是NP/PP和命名實體）。

>>> conll2002.categories() 
Traceback (most recent call last): 
    File "<stdin>", line 1, in <module> 
AttributeError: 'ConllChunkCorpusReader' object has no attribute 'categories' 

所以你的問題的簡短答案是，你不能在conll2002語料庫上訓練情感分析器。