2010-05-30 86 views
7

我有很多正在觸及HardDeadlineExceededError的任務/ servlet,這些任務正在讓一切都處於「仍在執行」狀態。用於處理的任務策略HardDeadlineExceededError

正在完成的工作可以輕鬆超過29秒的閾值。

我試圖抓住DeadlineExceededException和基礎例外,以保存退出 狀態,但這些都不例外處理程序被抓的......

有沒有一種方法來確定哪些任務是在隊列或正在執行?

有沒有其他的策略來處理這種情況?

我正在處理的情況記錄在"The Request Timer"標題下。

// task handler for retrieving information from external web services 
protected void doPost(HttpServletRequest req, HttpServletResponse resp) 
throws ServletException, IOException { 

    String taskRetryCountParam = req.getParameter("X-AppEngine-TaskRetryCount"); 
    int taskRetryCount = (taskRetryCountParam==null) ? 0 : Integer.parseInt(taskRetryCountParam); 
      // look up the persistent 'task' and mark it as 'running' 

    logger.info(this.getClass().getName() + ".doPost("+ taskId + ") retryCount=" + taskRestryCount); 


    // Do lots of heavy lifting here 
    // like calling external web services using URL fetch service 
      // and saving the contents into our database. 

      // look up the persistent 'task' and mark it as 'completed' 

    } catch (DeadlineExceededException deadline) { 
     // got this deadline exception 
        // look up the persistent 'task' and mark it as 'errored - try again' 
     logger.warning("DeadlineExceeded Exception while loading content " + deadline.getMessage()); 
     resp.setStatus(HttpServletResponse.SC_REQUEST_TIMEOUT); 

     } 
    } catch (Exception unknown) { 
     // got some unknown exception 
        // look up the persistent 'task' and mark it as 'errored - cancelled' 
     logger.severe("General Exception while loading content exception:" + unknown.getMessage()); 
     resp.setStatus(HttpServletResponse.SC_OK); 

    } 
} 

這裏是日誌文件條目,當我進入這個情況......看來,我的數據庫事務正在當談到時間太長。

W 05-30 12:42PM 09.535 
    Error for /loadstatus 
    com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC. 
    at java.lang.Object.wait(Native Method) 
    at java.lang.Object.wait(Object.java:443) 
    at java.util.concurrent.TimeUnit.timedWait(Unknown Source) 
    at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60) 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326) 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217) 
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131) 
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43) 
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) 
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58) 
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42) 
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56) 
    at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66) 
    at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61) 
    at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88) 
    at org.datanucleus.transaction.Transaction.commit(Transaction.java:149) 
    at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95) 
    at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390) 
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258) 
    at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83) 
    at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) 
    <snip> 
    C 05-30 12:42PM 09.629 
    Uncaught exception from servlet 
    com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC. 
    at java.lang.Object.wait(Native Method) 
    at java.lang.Object.wait(Object.java:443) 
    at java.util.concurrent.TimeUnit.timedWait(Unknown Source) 
    at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60) 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326) 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217) 
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131) 
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43) 
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) 
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58) 
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42) 
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56) 
    at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66) 
    at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61) 
    at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88) 
    at org.datanucleus.transaction.Transaction.commit(Transaction.java:149) 
    at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95) 
    at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390) 
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258) 
    at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83) 
    at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) 
    <snip> 
    W 05-30 12:42PM 09.644 
    A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104) 
+2

您應該可以捕捉到[DeadlineExceededException](http://code.google.com/appengine/docs/java/runtime.html#The_Request_Timer)。你可以發佈你的一些代碼嗎? – 2010-05-30 21:24:12

+0

@Matthew:我沒有做GAE,但是'HardDeadlineExceededError'聽起來不像是一個'Exception'。 – BalusC 2010-05-31 00:16:12

+0

@巴洛斯,我沒有說這是。 'DeadlineExceededException'和'HardDeadlineExceededError'是兩回事。前者可以被抓到。後者不能,如果你花費大約一秒鐘的時間來處理前者,則拋出後者。 – 2010-05-31 00:28:35

回答

3

http://groups.google.com/group/google-appengine-java/msg/e3fd2b621bb96013

HDEEs可以不用DEE,如果它在你自己的代碼恰好被拋出。
通常最耗時的事情是等待
的API調用返回,因此這裏的超時將導致API調用停止,並且 DEE。所以如果你不經常打電話給API,你可以直接打HDEE。

我也有長時間運行的任務迭代通過數據處理
並存儲結果。我使用了一個迭代器,它在20秒後停止返回結果
,並保存最後處理的對象,然後將
踢出新任務以繼續處理。

我原來的解決方案抓住了DEE,然後清理,但這個
停止工作可靠。

+0

感謝您輸入Marc。你是否抓住了錯誤並從它或其他東西推斷HDEE? – Stevko 2010-06-02 02:40:20

+1

您無法捕捉到HDEE錯誤。不是因爲你不能添加這樣的catch(你可以),而是你的代碼被停止了。爲此,應該在HDEE前半秒拋出DEE,但如果你在自己的代碼中,這不起作用。 – Marc 2010-06-02 05:44:39