2017-02-27 234 views
0

用下面的有效載荷在Python功能python elasticsearch helpers.bulk錯誤地失敗了嗎? (腳本或文檔丟失)

[{'_id': '979507',                                   [15/1871] 
    '_index': 'follow-search-alias', 
    '_op_type': 'update', 
    '_type': 'follow', 
    'script': {'inline': 'ctx._source.followers += follower', 
    'lang': 'groovy', 
    'params': {'follower': ['3054805']}}}, 
{'_id': '979507', 
    '_index': 'follow-search-alias', 
    '_op_type': 'update', 
    '_type': 'follow', 
    'script': {'inline': 'ctx._source.following += user_being_followed', 
    'lang': 'groovy', 
    'params': {'user_being_followed': []}}}, 
{'_id': '3054805', 
    '_index': 'follow-search-alias', 
    '_op_type': 'update', 
    '_type': 'follow', 
    'script': {'inline': 'ctx._source.followers += follower', 
    'lang': 'groovy', 
    'params': {'follower': []}}}, 
{'_id': '3054805', 
    '_index': 'follow-search-alias', 
    '_op_type': 'update', 
    '_type': 'follow', 
    'script': {'inline': 'ctx._source.following += user_being_followed', 
    'lang': 'groovy', 
    'params': {'user_being_followed': ['979507']}}}] 

我與Python的elasticsearch以下錯誤,當我使用helpers.bulk()

RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;4: script or doc is missing;') 

究竟如何能這是?此數組中的每個元素都有一個腳本標記。該函數本身一直在其他許多情況下工作,只是不是這個和其他幾個選擇。

在shell中手動運行它的工作,但不是在這個函數中?

功能如下:

@classmethod 
def add_follows(cls, follows): 
    docs = [] 
    user_ids = [f.follower_id for f in follows] + [f.followed_id for f in follows] 
    users = User.query.filter(User.id.in_(user_ids)).all() 
    valid_user_ids = set([u.id for u in users]) 

    grouped_follows = {} 

    for follow in follows: 
     if (follow.follower_id not in valid_user_ids) or (follow.followed_id not in valid_user_ids): 
      continue 
     if not follow.follower_id in grouped_follows: 
      grouped_follows[follow.follower_id] = { 
       'followers': [], 
       'following': [] 
       } 
     if not follow.followed_id in grouped_follows: 
      grouped_follows[follow.followed_id] = { 
       'followers': [], 
       'following': [] 
       } 

     grouped_follows[follow.follower_id]['following'].append(str(follow.followed_id)) 
     grouped_follows[follow.followed_id]['followers'].append(str(follow.follower_id)) 

    for user_id, data in grouped_follows.items(): 
     follower_action = { 
      '_index': FSC.FOLLOW_SEARCH_INDEX_NAME, 
      '_type': FSC.FOLLOW_SEARCH_MAPPING_NAME, 
      '_id': str(user_id), 
      '_op_type': 'update', 
      'script': { 
       'inline': 'ctx._source.followers += follower', 
       'params': { 
        'follower': data['followers'] 
        }, 
       'lang': 'groovy' 
      } 
     } 

     followed_action = { 
      '_index': FSC.FOLLOW_SEARCH_INDEX_NAME, 
      '_type': FSC.FOLLOW_SEARCH_MAPPING_NAME, 
      '_id': str(user_id), 
      '_op_type': 'update', 
      'script': { 
       'inline': 'ctx._source.following += user_being_followed', 
       'params': { 
        'user_being_followed': data['following'] 
        }, 
       'lang': 'groovy' 
      } 
     } 

     docs += [follower_action, followed_action] 

    print docs 
    if docs: 
     helpers.bulk(es, docs, request_timeout=300) 

我目前在shell中運行此。我甚至把兩行:

global payload 
payload = docs 

而在shell中運行

# after the above function fails 
In [96]: helpers.bulk(es, payload) 
Out[96]: (4, []) 

所以它的工作原理?相同的負載?同樣的功能?剛纔它的功能在外?鑑於這些問題,我甚至無法相信這個圖書館將在生產中工作。

回答

0

這是非常難以調試沒有附加信息,你可以啓用記錄日誌記錄器和粘貼輸出的地方,因爲它應該包含完整的信息,包括髮送到elasticsearch的實際數據?謝謝