如何使用logstash變異或紅寶石濾波

Question

我有以下的JSON語法

{"result": { 
    "entities": { 
     "SERVICE-CCC89FB0A922657A": "service1", 
     "SERVICE-D279F46CD751424F": "service2", 
     "SERVICE-7AB760E70FCDCA18": "service3", 
    }, 
    "dataPoints": { 
     "SERVICE-CCC89FB0A922657A": [ 
      [ 
       1489734240000, 
       1101.0 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ], 
     "SERVICE-7AB760E70FCDCA18": [ 
      [ 
       1489734240000, 
       4080800.5470588235 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ], 
     "SERVICE-D279F46CD751424F": [ 
      [ 
       1489734240000, 
       26677.695652173912 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ] 
    } 
}, 
"@timestamp": "2017-03-17T07:05:37.531Z", 
"data": "data", 
"@version": "1" 
} 

我想改變下列輸入它elasticsearch。

{"@timestamp": "2017-03-17T07:05:37.531Z", 
"data": "data", 
"@version": "1", 
    "data" : { 
     "service1",: [ 
      [ 
       1489734240000, 
       1101.0 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ], 
     "service3" : [ 
      [ 
       1489734240000, 
       4080800.5470588235 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ], 
     "service2": [ 
      [ 
       1489734240000, 
       26677.695652173912 
      ], 
      [ 
       1489734300000, 
       null 
      ] 
     ] 
    } 
} 

這是當前logstash conf文件的內容。

input { 
    http_poller { 
     urls => { 
      test => { 
       method => get 
       url => "https://xxxx.com" 
       headers => { 
        Accept => "application/json" 
       } 
      } 
     } 
     request_timeout => 60 
     schedule => { every => "60s" } 
     codec => "plain" 
    } 
} 

filter { 
    json{ 
     source => "message" 
     remove_field => ["[result][aggregationType]","message"] 
    } 

# translate{ 
# } 

# mutate{ 
# } 

# ruby{ 
# } 

} 

output { 
    stdout { 
     codec => rubydebug { 
     #metadata => true 
     } 
    } 
    elasticsearch { 
     hosts => ["http://192.168.0.36:9200"] 
    } 
} 

我剛剛使用elasticsearch，我不知道如何實現什麼過濾器使用。 我不知道是否有可能實現mutate過濾器重命名的內容。 或者我應該用ruby過濾器實現代碼嗎？

實體可能會與ruby過濾器一起排列，以匹配dataPoints的SERVICE- *。 但是，很難處理Ruby代碼。

我希望你能幫助我。

謝謝。

Answer 1

這裏有幾個過濾器用於logstash ...

https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html