2013-04-09 53 views
14

連續獲取下面的警告消息。不知道該做什麼。看到一些相關的帖子要求增加文件描述符的數量。彈性搜索警告的打開文件太多

如何做到這一點?

即使我現在增加,我是否會在添加新索引時遇到同樣的問題。 (目前與約400個指數,6個碎片和1個副本一起工作)。指數的數量往往增長得更多。節點API的

[03:58:24,165][WARN ][cluster.action.shard  ] [node1] received shard failed for [index9][2], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index9][2] failed recovery]; nested: EngineCreationFailureException[[index9][2] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index9/2/index/segments_1 (Too many open files)]; ]] 
[03:58:24,166][WARN ][cluster.action.shard  ] [node1] received shard failed for [index15][0], node[node_hash2], [P], s[INITIALIZING], reason [Failed to create shard, message [IndexShardCreationException[[index15][0] failed to create shard]; nested: IOException[directory '/data/elasticsearch/whatever/nodes/0/indices/index15/0/index' exists and is a directory, but cannot be listed: list() returned null]; ]] 
[03:58:24,195][WARN ][cluster.action.shard  ] [node1] received shard failed for [index16][3], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index16][3] failed recovery]; nested: EngineCreationFailureException[[index16][3] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index16/3/index/segments_1 (Too many open files)]; ]] 
[03:58:24,196][WARN ][cluster.action.shard  ] [node1] received shard failed for [index17][0], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index17][0] failed recovery]; nested: EngineCreationFailureException[[index17][0] failed to open reader on writer]; nested: FileNotFoundException[/data/elasticsearch/whatever/nodes/0/indices/index17/0/index/segments_1 (Too many open files)]; ]] 
[03:58:24,198][WARN ][cluster.action.shard  ] [node1] received shard failed for [index21][4], node[node_hash3], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[index21][4] failed recovery]; nested: EngineCreationFailureException[[index21][4] failed to create engine]; nested: LockReleaseFailedException[Cannot forcefully unlock a NativeFSLock which is held by another indexer component: /data/elasticsearch/whatever/nodes/0/indices/index21/4/index/write.lock]; ]] 

輸出

curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true' 

{ 
    "ok" : true, 
    "cluster_name" : "whatever", 
    "nodes" : { 
    "node_hash1" : { 
     "name" : "node1", 
     "transport_address" : "transportip1", 
     "hostname" : "myhostip1", 
     "version" : "0.20.4", 
     "http_address" : "httpip1", 
     "attributes" : { 
     "data" : "false", 
     "master" : "true" 
     }, 
     "os" : { 
     "refresh_interval" : 1000, 
     "available_processors" : 8, 
     "cpu" : { 
      "vendor" : "Intel", 
      "model" : "Xeon", 
      "mhz" : 2133, 
      "total_cores" : 8, 
      "total_sockets" : 8, 
      "cores_per_socket" : 16, 
      "cache_size" : "4kb", 
      "cache_size_in_bytes" : 4096 
     }, 
     "mem" : { 
      "total" : "7gb", 
      "total_in_bytes" : 7516336128 
     }, 
     "swap" : { 
      "total" : "30gb", 
      "total_in_bytes" : 32218378240 
     } 
     }, 
     "process" : { 
     "refresh_interval" : 1000, 
     "id" : 26188, 
     "max_file_descriptors" : 16384 
     } 
    }, 
    "node_hash2" : { 
     "name" : "node2", 
     "transport_address" : "transportip2", 
     "hostname" : "myhostip2", 
     "version" : "0.20.4", 
     "attributes" : { 
     "master" : "false" 
     }, 
     "os" : { 
     "refresh_interval" : 1000, 
     "available_processors" : 4, 
     "cpu" : { 
      "vendor" : "Intel", 
      "model" : "Xeon", 
      "mhz" : 2400, 
      "total_cores" : 4, 
      "total_sockets" : 4, 
      "cores_per_socket" : 32, 
      "cache_size" : "20kb", 
      "cache_size_in_bytes" : 20480 
     }, 
     "mem" : { 
      "total" : "34.1gb", 
      "total_in_bytes" : 36700303360 
     }, 
     "swap" : { 
      "total" : "0b", 
      "total_in_bytes" : 0 
     } 
     }, 
     "process" : { 
     "refresh_interval" : 1000, 
     "id" : 24883, 
     "max_file_descriptors" : 16384 
     } 
    }, 
    "node_hash3" : { 
     "name" : "node3", 
     "transport_address" : "transportip3", 
     "hostname" : "myhostip3", 
     "version" : "0.20.4", 
     "attributes" : { 
     "master" : "false" 
     }, 
     "os" : { 
     "refresh_interval" : 1000, 
     "available_processors" : 4, 
     "cpu" : { 
      "vendor" : "Intel", 
      "model" : "Xeon", 
      "mhz" : 2666, 
      "total_cores" : 4, 
      "total_sockets" : 4, 
      "cores_per_socket" : 16, 
      "cache_size" : "8kb", 
      "cache_size_in_bytes" : 8192 
     }, 
     "mem" : { 
      "total" : "34.1gb", 
      "total_in_bytes" : 36700303360 
     }, 
     "swap" : { 
      "total" : "0b", 
      "total_in_bytes" : 0 
     } 
     }, 
     "process" : { 
     "refresh_interval" : 1000, 
     "id" : 25328, 
     "max_file_descriptors" : 16384 
     } 
    } 
    } 
+0

@Vamshi克里希納:如果你可以在這裏幫助http://stackoverflow.com/questions/28082825/elasticsearch-log-file-huge-size-performance-degradation – inquisitive 2015-01-22 06:54:20

+0

這不是一個警告。這是一個錯誤。您無法打開文件。 – EJP 2015-10-08 21:36:04

回答

12

如何增加允許打開的文件的最大數量在你的Linux發行略有依賴。下面是爲Ubuntu和CentOS一些指令:

http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/ http://pro.benjaminste.in/post/318453669/increase-the-number-of-file-descriptors-on-centos-and

的elasticsearch文檔建議的最大文件限制設置爲32K或64K。既然你已經達到了16K,並且已經達到了極限,我可能會把它設置得更高;像128k的東西。請參閱:http://www.elasticsearch.org/guide/reference/setup/installation/

正在增加打開文件的數量和重啓elasticsearch後,你會想驗證它的工作重新運行你所提到的curl命令:

curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true' 

當你添加更多指數(連同更多文檔),您還將看到elasticsearch跟蹤增加的文件數量。如果您注意到所有指示和文檔的性能下降,您可以嘗試向集羣添加新節點:http://www.elasticsearch.org/guide/reference/setup/configuration/ - 因爲您已經擁有分片的複製配置,所以這應該是一個相對輕鬆的過程。

+0

嗨@丹Noble感謝您的信息。我的ulimit已經設置爲65535.彈性搜索怎麼只取16384?我需要更改配置文件中的設置以獲取由OS設置的ulimit ...嗎? – 2013-04-09 17:30:02

+0

是的。你解決了嗎? – 2013-04-12 19:35:15

+1

糟糕...對不起,我接受了答案。但忘了提及相同的。是的,我必須在操作系統級別更改它以獲取更新的限制。您的評論幫助了很多。謝謝@Dan – 2013-04-15 05:57:55

0
  1. Stop ElasticSearch。如果你從像 (bin/elasticsearch)這樣的命令開始,那麼請具體設置堆,而 開始。對於前者,我使用16GB盒子,所以我的命令是

    a。 bin/elasticsearch -Xmx8g -Xms8g

    b。轉到配置(elasticsearch /配置/ elasticsearch.yml),並確保

    bootstrap.mlockall:真

    ℃。增加ulimits -Hnulimits -Sn多於200000

  2. 如果作爲服務啓動,然後執行以下操作

    一個。導出ES_HEAP_SIZE = 10g b。轉到config(/etc/elasticsearch/elasticsearch.yml)並確保

    bootstrap.mlockall:true c。增加ulimits -Hnulimits -Sn多於200000

請確保您輸入的大小不在於你是否開始把它作爲服務堆的50%以上,或從命令行