mergnig名詞短語塊的POS標籤

Question

我的問題類似於這個question。在spacy中，我可以分別進行詞性標註和名詞短語標識，例如

import spacy 
nlp = spacy.load('en') 
sentence = 'For instance , consider one simple phenomena : 
      a question is typically followed by an answer , 
      or some explicit statement of an inability or refusal to answer .' 
token = nlp(sentence) 
token_tag = [(word.text, word.pos_) for word in token] 

輸出的樣子：

[('For', 'ADP'), 
('instance', 'NOUN'), 
(',', 'PUNCT'), 
('consider', 'VERB'), 
('one', 'NUM'), 
('simple', 'ADJ'), 
('phenomena', 'NOUN'), 
...] 

對於名詞短語或塊，我可以得到noun_chunks這是詞的一大塊如下：

[nc for nc in token.noun_chunks] # [instance, one simple phenomena, an answer, ...] 

我想知道是否有是一種基於noun_chunks對POS標籤進行聚類的方式，以便我得到輸出爲

[('For', 'ADP'), 
('instance', 'NOUN'), # or NOUN_CHUNKS 
(',', 'PUNCT'), 
('one simple phenomena', 'NOUN_CHUNKS'), 
...] 

Answer 1

我想出瞭如何做到這一點。基本上，我們可以得到啓動和名詞短語符的結束位置如下：

noun_phrase_position = [(s.start, s.end) for s in token.noun_chunks] 
noun_phrase_text = dict([(s.start, s.text) for s in token.noun_chunks]) 
token_pos = [(i, t.text, t.pos_) for i, t in enumerate(token)] 

然後我用這個solution相結合，以合併基礎上start的token_pos名單，stop位置

result = [] 
for start, end in noun_phrase_position: 
    result += token_pos[index:start] 
    result.append(token_pos[start:end]) 
    index = end 

result_merge = [] 
for i, r in enumerate(result): 
    if len(r) > 0 and isinstance(r, list): 
     result_merge.append((r[0][0], noun_phrase_text.get(r[0][0]), 'NOUN_PHRASE')) 
    else: 
     result_merge.append(r) 

輸出

[(1, 'instance', 'NOUN_PHRASE'), 
(2, ',', 'PUNCT'), 
(3, 'consider', 'VERB'), 
(4, 'one simple phenomena', 'NOUN_PHRASE'), 
(7, ':', 'PUNCT'), 
(8, 'a', 'DET'), ...