使用MongoDB Broker的芹菜

Question

使用mongodb代理作爲後端工作來獲取Celery設置（遵循基本教程）。以下列出在官方文檔中的配置原則，我celeryconfig.py的設置如下：

CELERY_RESULT_BACKEND = "mongodb" 

BROKER_BACKEND = "mongodb" 
BROKER_URL = "mongodb://user:pass@subdomain.mongolab.com:123456/testdb" 

CELERY_MONGODB_BACKEND_SETTINGS = { 
    "host":"subdomain.mongolab.com", 
    "port":123456, 
    "database":"testdb", 
    "taskmeta_collection":"taskmeta", 
    "user":"user", 
    "pass":"pass", 
} 
CELERY_IMPORTS = ("tasks",) 

運行celeryd與--loglevel=INFO返回以下異常，對原產於pymongo但通過兩個海帶，芹菜冒泡。

Traceback (most recent call last): 
    File "/usr/local/lib/python2.7/dist-packages/celery/worker/__init__.py", line 230, in start 
    component.start() 
    File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 338, in start 
    self.reset_connection() 
    File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 596, in reset_connection 
    on_decode_error=self.on_decode_error) 
    File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 335, in get_task_consumer 
    **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/kombu/compat.py", line 187, in __init__ 
    super(ConsumerSet, self).__init__(self.backend, queues, **kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 285, in __init__ 
    self.declare() 
    File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 295, in declare 
    queue.declare() 
    File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 388, in declare 
    self.queue_declare(nowait, passive=False) 
    File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 408, in queue_declare 
    nowait=nowait) 
    File "/usr/local/lib/python2.7/dist-packages/kombu/transport/virtual/__init__.py", line 380, in queue_declare 
    return queue, self._size(queue), 0 
    File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 74, in _size 
    return self.client.messages.find({"queue": queue}).count() 
    File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 171, in client 
    self._client = self._open() 
    File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 97, in _open 
    mongoconn = Connection(host=conninfo.hostname, port=conninfo.port) 
    File "/usr/local/lib/python2.7/dist-packages/pymongo/connection.py", line 325, in __init__ 
    nodes.update(uri_parser.split_hosts(entity, port)) 
    File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 198, in split_hosts 
    nodes.append(parse_host(entity, default_port)) 
    File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 127, in parse_host 
    raise ConfigurationError("Reserved characters such as ':' must be " 
ConfigurationError: Reserved characters such as ':' must be escaped according RFC 2396. An IPv6 address literal must be enclosed in '[' and ']' according to RFC 2732. 

講講方式芹菜處理mongouri不正確的編碼，因爲它是內pymongo是引發此錯誤的URI解析器。我試圖轉義uri字符串中的:字符，但所有這些都實現了將傳輸重置爲具有損壞連接字符串的默認AMQP。

amqp://guest@localhost:5672/mongodb\http://user\:password@subdomain.mongolab.com\:29217/testdb 

這顯然是不正確的。

我已經嘗試使用r作爲原始字符串在配置中輸入uri，但沒有任何更改。我知道這種連接配置自2.4（我使用2.5.1，pymongo 2.1.1）以來一直在Celery支持，官方文檔都引用它作爲連接到mongodb代理的首選方法。

這可能是一個錯誤，也許與最新的pymongo構建不兼容？如果這種方法不起作用，那麼如何將任務隊列附加到副本集，因爲我認爲這些必須通過使用參數?replicaSet在mongouri中傳遞。

我應該注意的是，我寧願不轉而使用RabbitMQ代理，因爲Mongo已經在應用程序的堆棧中，它似乎更直觀地使用已經存在的代碼。如果有一個具體的原因，爲什麼Mongo在這個目的上效率不高（每天的任務量相對較小），我很想知道！提前致謝。

Answer 1

我認爲這是一個錯誤。 Celery將主機名而不是server_uri傳遞給kombu，從而導致此問題。在追蹤代碼之後，我發現下面的conf在修復它之前繞過了這個bug。

CELERY_RESULT_BACKEND = 'mongodb' 
BROKER_HOST = "subdomain.mongolab.com" 
BROKER_PORT = 123456 
BROKER_TRANSPORT = 'mongodb' 
BROKER_VHOST = 'testdb' 
CELERY_IMPORTS = ('tasks',) 
CELERY_MONGODB_BACKEND_SETTINGS = { 
     'host': 'subdomain.mongolab.com', 
     'port': 123456, 
     'database': 'testdb', 
     'user': user, 
     'password': password, 
     'taskmeta_collection': 'teskmeta' 
     } 

只是重複配置。

Answer 2

請問如果去掉「用戶」它幫「通」，「端口」，然後從CELERY_MONGODB_BACKEND_SETTINGS字典「數據庫」，然後執行：

BROKER_URL = "mongodb://user:pass@subdomain.mongolab.com:123456/testdb" 

CELERY_MONGODB_BACKEND_SETTINGS = { 
    "host":BROKER_URL, 
    "taskmeta_collection":"taskmeta", 
}