2016-04-27 58 views
0

面臨從csv轉換某些值的問題。 如果我只聲明一個轉換例如'id'=>'整數'它工作正常。 如果我再添轉換像它下面的崩潰配置....Logastash CSV過濾器|轉換

input { 
     tcp { 
      port => 5000 
     } 
     file { 
       path => '/tmp/logstash/databases/1.csv' 
       type => 'test' 
       start_position => 'beginning' 
      } 
    } 

    filter { 
     csv { 
      columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time'] 
      separator => ',' 
      convert => {'Accuracy' => 'float','id'=>'integer'} 
     } 
    } 

    output { 
     elasticsearch { 
       hosts => 'elasticsearch:9200' 
     } 
     stdout{ 

     } 
    } 

錯誤:

{:timestamp=>"2016-04-27T18:49:51.881000+0000", 
:message=>"fetched an invalid config", 
:config=>"input {\n\ttcp {\n\t\tport => 5000\n\t}\n\tfile {\n 
path => '/tmp/logstash/databases/1.csv'\n   
type => 'test'\n   start_position => 'beginning'\n 
}\n}\n\nfilter {\n csv {\n   
columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time']\n   
separator => ','\n\t\t\t\tconvert => {'Accuracy' => 'float','id'=>'integer'}\n }\n}\n\noutput {\n\telasticsearch {\n\t\t\thosts => 'elasticsearch:9200'\n\t}\n\tstdout{\n\n\t}\n}\n\n", 
:reason=>"Expected one of #, {, } at line 16, column 38 (byte 391) after filter {\n csv {\n  
columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time']\n   
separator => ','\n\t\t\t\tconvert => {'Accuracy' => 'float'", :level=>:error} 
+0

UPDATE:與發生變異的作品轉換吧! –

回答

-1

你可以使用它作爲:

... 
convert => { 
    'Accuracy' => 'float' 
    'Id' => 'integer' 
} 
... 

之間沒有逗號轉換。

您還可以使用---- configtest選項來檢查你的配置文件中的錯誤:

λ bin/logstash -f example.conf --configtest 
+0

謝謝alpert - 不再錯誤。但不幸的是,字符串沒有被轉換:/精度被保存爲例如30.0 –

+0

要更清楚:在kibana內,它顯示「字符串」作爲準確性的類型 –

+0

刪除由logstash創建的索引和索引模板。刪除標題表單csv。它會創建數字格式例外。重新索引您的數據。這對我有效。另外'id'必須是'Id'。 – alpert