芹菜工人多次接收同一個id的任務

我有一個芹菜django項目。我看到一些工人不止一次接受任務使用同一個ID的問題。這裏是日誌的一個片段：芹菜工人多次接收同一個id的任務

9238 [2015-07-03 10:25:03,310: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[72171e05-665b-4730-8499-3da821838935] succeeded in 0.3505859375s: None 
9239 [2015-07-03 10:25:03,310: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[70d6a7d3-3cb4-4e01-8d04-4d084e36bdfb] 
9240 [2015-07-03 10:25:03,444: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[08b6bb3f-6a8a-43a1-992d-e16ceae76e1d] succeeded in 0.750570058823s: None 
9241 [2015-07-03 10:25:03,445: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[8df19680-ba43-4a43-8316-cfbb9bb95e30] 
9242 [2015-07-03 10:25:03,633: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[72171e05-665b-4730-8499-3da821838935] succeeded in 0.30401802063s: None 
9243 [2015-07-03 10:25:03,639: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[70d6a7d3-3cb4-4e01-8d04-4d084e36bdfb] 
9244 [2015-07-03 10:25:03,746: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[8df19680-ba43-4a43-8316-cfbb9bb95e30] succeeded in 0.298280000687s: None 
9245 [2015-07-03 10:25:03,747: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[8df19680-ba43-4a43-8316-cfbb9bb95e30] 
9246 [2015-07-03 10:25:03,894: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[70d6a7d3-3cb4-4e01-8d04-4d084e36bdfb] succeeded in 0.563042879105s: None 
9247 [2015-07-03 10:25:03,895: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[3b2d1317-4ff1-4279-84ae-696d35997358] 
9248 [2015-07-03 10:25:04,120: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[8df19680-ba43-4a43-8316-cfbb9bb95e30] succeeded in 0.332364082336s: None 
`enter code here`9249 [2015-07-03 10:25:04,122: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[3b2d1317-4ff1-4279-84ae-696d35997358] 
9250 [2015-07-03 10:25:04,218: INFO/MainProcess] Task ssrweb.execution.tasks.get_script_results[70d6a7d3-3cb4-4e01-8d04-4d084e36bdfb] succeeded in 0.550192832947s: None 
9251 [2015-07-03 10:25:04,219: INFO/MainProcess] Got task from broker: ssrweb.execution.tasks.get_script_results[e7854ef6-6dbf-4840-87bb-699e32d168ce]

70d6a7d3-3cb4-4e01-8d04-4d084e36bdfb重複上線9239，9243，和上線9246和9250

這些任務會顯示執行

看到Id作爲django管理頁面???中的問號。

我使用的（我知道這些都是舊版本）：

host:230> ./manage.py celeryd --version 
1.3.1 
celery 2.5.3 
django-celery 2.4.2

我這是怎麼開始的工人和celerybeat：

1 #!/bin/sh 
2 
3 # not used 
4 #./manage.py celeryd -l info -E -B --scheduler=djcelery.schedulers.DatabaseScheduler 
5 #./manage.py celeryd -l info -E -B --scheduler=djcelery.schedulers.DatabaseScheduler --pidfile=/opt/ssr/var/run/celery/celery.pid 
6 
7 CELERY_LOG_DIR="/opt/ssr/var/log/celery" 
8 CELERY_RUN_DIR="/opt/ssr/var/run/celery" 
9 CELERY_SCHEDULER="djcelery.schedulers.DatabaseScheduler" 
10 LANG=en_US.UTF8 
11 
12 mkdir -p $CELERY_LOG_DIR 
13 mkdir -p $CELERY_RUN_DIR 
14 
15 # Workers for periodic tasks in execution and bvt 
16 ./manage.py celeryd_multi start w1 -c 3 -l info -E -Q p_action --logfile=$CELERY_LOG_DIR/w1.log --pidfile=$CELERY_RUN_DIR/w1.pid 
17 ./manage.py celeryd_multi start w2 -c 3 -l info -E -Q p_action,p_check,p_cleanup,p_notify --logfile=$CELERY_LOG_DIR/w2.log --pidfile=$CELERY_RUN_DIR/w2.pid 
18 
19 # Workers for ondemand tasks in execution and bvt 
20 ./manage.py celeryd_multi start w3 -c 3 -l info -E -Q od_action --logfile=$CELERY_LOG_DIR/w3.log --pidfile=$CELERY_RUN_DIR/w3.pid 
21 ./manage.py celeryd_multi start w4 -c 3 -l info -E -Q od_action,od_notify,od_check --logfile=$CELERY_LOG_DIR/w4.log --pidfile=$CELERY_RUN_DIR/w4.pid 
22 
23 # Workers for the default queue 
24 ./manage.py celeryd_multi start w5 -c 2 -l info -E -Q default --logfile=$CELERY_LOG_DIR/w5.log --pidfile=$CELERY_RUN_DIR/w5.pid 
25 ./manage.py celeryd_multi start w6 -c 2 -l info -E -Q default --logfile=$CELERY_LOG_DIR/w6.log --pidfile=$CELERY_RUN_DIR/w6.pid 
26 
27 # Celerybeat 
28 ./manage.py celerybeat -S $CELERY_SCHEDULER

切換此設置開/關沒求助：

#CELERY_ACKS_LATE = True 
#CELERYD_PREFETCH_MULTIPLIER = 1

我已經看到了問題here，但還沒有看到最終的解決辦法。

我也看過this one，但它也沒有一個決定性的解決方案，我想給我的確切上下文。

任何幫助，將不勝感激。

來源

2015-07-03 A. B

這也發生在我身上。
在我的tasks.py中，我有兩個periodic_tasks，其中一個比另一個花時間長，最初開始。

@periodic_tasks(run_every=timedelta(minutes=15)) 
def start_monitoring() ## This was taking some time to get started. 

@periodic_tasks(run_every=timedelta(minutes=5)) 
def if_script_no_run_then_run_script() ## This runs quickly but ended up spawning multiple scripts when spinning up the workers.

我最終把更快的periodic_tasks放在較慢的之前，這對我有效。

來源

2016-04-24 07:23:30

芹菜工人多次接收同一個id的任務

回答

相關問題