Logstash多行日誌文件的XML解析過濾

Question

我絕對新Logstash，我試圖分析我多logentries，是按以下格式

<log level="INFO" time="Wed May 03 08:25:03 CEST 2017" timel="1493792703368" host="host"> <msg><![CDATA[Method=GET URL=http://localhost (Vers=[Version], Param1=[param1], Param2=[param1]) Result(Content-Length=[22222], Content-Type=[text/xml; charset=utf-8]) Status=200 Times=TISP:1098/CSI:-/Me:1/Total:1099]]> </msg> </log>

你知道如何實現logstash過濾器配置能夠爲指數elasticsearch以下領域

時間，主持人，弗斯，參數1，參數2，TISP

非常感謝您

Answer 1

好的，我發現如何去做。這是我的pipeline.conf文件，它可以工作

input { 
     beats { 
       port => 5044 
     } 
} 

filter { 
     xml { 
       store_xml => false 
       source => "message" 
       xpath => [ 
       "/log/@level", "level", 
       "/log/@time", "time", 
       "/log/@timel", "unixtime", 
       "/log/@host", "host_org", 
       "/log/@msg", "msg", 
       "/log/msg/text()","msg_txt" 
       ] 
     } 

     grok { 
       break_on_match => false 
       match => ["msg_txt", "Param1=\[(?<param1>-?\w+)\]"] 
       match => ["msg_txt", "Param2=\[(?<param2>-?\w+)\]"] 
       match => ["msg_txt", "Vers=\[(?<vers>-?\d+\.\d+)\]"] 
       match => ["msg_txt", "TISP:(?<tisp>-?\d+)"] 
       match => [unixtime, "(?<customTime>-?\d+)"] 
     } 
     if "_grokparsefailure" in [tags] { 
       drop { } 
     } 

     mutate { 
       convert => { "tisp" => "integer" } 
     } 

     date { 
       match => [ "customTime", "UNIX_MS"] 
       target => "@timestamp" 
     } 
     if "_dateparsefailure" in [tags] { 
       drop { } 
     } 



} 

output { 
     elasticsearch { 
       hosts => "elasticsearch:9200" 
       user => user 
       password => passwd 
     } 
}