NLTK中的詞義消歧蟒Python

Question

我是NLTK Python的新手，我正在尋找一些可以對詞義進行消歧的示例應用程序。我在搜索結果中有很多算法，但沒有示例應用程序。我只想通過一個句子，並希望通過參考wordnet庫來了解每個單詞的含義。 謝謝

我在PERL中找到了一個類似的模塊。 http://marimba.d.umn.edu/allwords/allwords.html NLTK Python中是否有這樣的模塊？

Answer 1

是的，它可以在NLTK中使用wordnet模塊。 在您的文章中提到的工具中使用的相似性度量也存在於NLTK wordnet模塊中。

Answer 2

參考http://jaganadhg.freeflux.net/blog/archive/2010/10/16/wordnet-sense-similarity-with-nltk-some-basics.html

Answer 3

NLTK有API來訪問WORDNET。 Wordnet將單詞作爲同義詞。這會給你一些關於這個詞，它的上位詞，下位詞，根詞等的信息。

「Python文本處理與NLTK 2.0食譜」是一本好書，讓你開始瞭解NLTK的各種功能。閱讀，理解和實施很容易。另外，你可以看看其他論文（在NLTK領域之外），其中討論了使用維基百科進行詞義消歧。

Answer 4

是的，實際上，有NLTK團隊編寫的a book，其中有多個章節的分類，他們明確涵蓋how to use WordNet。您也可以從Safari購買本書的物理版本。

僅供參考：NLTK由自然語言編程學者編寫，用於他們的入門編程課程。

Answer 5

作爲一個實際的答案OP的要求，這裏有幾個WSD方法Python實現，在NLTK的同義詞集合（S）的形式返回感官，https://github.com/alvations/pywsd

它包括

• Lesk算法（包括原始萊斯克,改編Lesk和簡單Lesk）
• 基線算法（隨機感，第一感測，最頻繁的感應）

它可以用作例如：

#!/usr/bin/env python -*- coding: utf-8 -*- 

bank_sents = ['I went to the bank to deposit my money', 
'The river bank was full of dead fishes'] 

plant_sents = ['The workers at the industrial plant were overworked', 
'The plant was no longer bearing flowers'] 

print "======== TESTING simple_lesk ===========\n" 
from lesk import simple_lesk 
print "#TESTING simple_lesk() ..." 
print "Context:", bank_sents[0] 
answer = simple_lesk(bank_sents[0],'bank') 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

print "#TESTING simple_lesk() with POS ..." 
print "Context:", bank_sents[1] 
answer = simple_lesk(bank_sents[1],'bank','n') 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

print "#TESTING simple_lesk() with POS and stems ..." 
print "Context:", plant_sents[0] 
answer = simple_lesk(plant_sents[0],'plant','n', True) 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

print "======== TESTING baseline ===========\n" 
from baseline import random_sense, first_sense 
from baseline import max_lemma_count as most_frequent_sense 

print "#TESTING random_sense() ..." 
print "Context:", bank_sents[0] 
answer = random_sense('bank') 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

print "#TESTING first_sense() ..." 
print "Context:", bank_sents[0] 
answer = first_sense('bank') 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

print "#TESTING most_frequent_sense() ..." 
print "Context:", bank_sents[0] 
answer = most_frequent_sense('bank') 
print "Sense:", answer 
print "Definition:",answer.definition 
print 

[OUT]：

======== TESTING simple_lesk =========== 

#TESTING simple_lesk() ... 
Context: I went to the bank to deposit my money 
Sense: Synset('depository_financial_institution.n.01') 
Definition: a financial institution that accepts deposits and channels the money into lending activities 

#TESTING simple_lesk() with POS ... 
Context: The river bank was full of dead fishes 
Sense: Synset('bank.n.01') 
Definition: sloping land (especially the slope beside a body of water) 

#TESTING simple_lesk() with POS and stems ... 
Context: The workers at the industrial plant were overworked 
Sense: Synset('plant.n.01') 
Definition: buildings for carrying on industrial labor 

======== TESTING baseline =========== 
#TESTING random_sense() ... 
Context: I went to the bank to deposit my money 
Sense: Synset('deposit.v.02') 
Definition: put into a bank account 

#TESTING first_sense() ... 
Context: I went to the bank to deposit my money 
Sense: Synset('bank.n.01') 
Definition: sloping land (especially the slope beside a body of water) 

#TESTING most_frequent_sense() ... 
Context: I went to the bank to deposit my money 
Sense: Synset('bank.n.01') 
Definition: sloping land (especially the slope beside a body of water) 

Answer 6

最近，pywsd代碼的一部分已被移植到的NLTK的最新版本中模塊，嘗試：

>>> from nltk.wsd import lesk 
>>> sent = 'I went to the bank to deposit my money' 
>>> ambiguous = 'bank' 
>>> lesk(sent, ambiguous) 
Synset('bank.v.04') 
>>> lesk(sent, ambiguous).definition() 
u'act as the banker in a game or in gambling' 

爲了獲得更好的性能WSD，而不是使用的NLTK模塊pywsd庫。一般來說，從pywsd的simple_lesk()比NLTK的lesk好。當我有空時，我會盡量更新NLTK模塊。

---

在迴應克里斯斯賓塞的評論，請注意的Lesk算法的限制。我只是簡單地給出一個算法的準確實現。這不是一個銀彈，http://en.wikipedia.org/wiki/Lesk_algorithm

還要注意的是，雖然：

lesk("My cat likes to eat mice.", "cat", "n") 

不給你正確的答案，你可以使用pywsd實施max_similarity()：

>>> from pywsd.similarity import max_similiarity 
>>> max_similarity('my cat likes to eat mice', 'cat', 'wup', pos='n').definition 
'feline mammal usually having thick soft fur and no ability to roar: domestic cats; wildcats' 
>>> max_similarity('my cat likes to eat mice', 'cat', 'lin', pos='n').definition 
'feline mammal usually having thick soft fur and no ability to roar: domestic cats; wildcats' 

@ Chris，如果你想要一個python setup.py，只是做一個禮貌的請求，我會寫它...