如何使用實驗性API將大文件寫入Blobstore？

我有兩難困境..我使用tipfy作爲框架上傳scribd store和blobstore中的文件。我有webform的行動不是由blobstore.create_upload_url（我只是使用url_for（'myhandler'））創建的。我這樣做是因爲如果我使用blobstore處理程序解析POST響應，並且我不能使用普通的python-scribd api將文件上傳到scribd商店。現在我已經工作scribd保護程序：如何使用實驗性API將大文件寫入Blobstore？

class UploadScribdHandler(RequestHandler, BlobstoreUploadMixin): 
    def post(self): 
     uploaded_file = self.request.files.get('upload_file') 
     fname = uploaded_file.filename.strip() 
     try: 
      self.post_to_scribd(uploaded_file, fname) 
     except Exception, e: 
      # ... get the exception message and do something with it 
      msg = e.message 
      # ... 
     # reset the stream to zero (beginning) so the file can be read again 
     uploaded_file.seek(0) 
     #removed try-except to see debug info in browser window 
     # Create the file 

     file_name = files.blobstore.create(_blobinfo_uploaded_filename=fname) 
     # Open the file and write to it 
     with files.open(file_name, 'a') as f: 
      f.write(uploaded_file.read()) 
     # Finalize the file. Do this before attempting to read it.  
     files.finalize(file_name) 
     # Get the file's blob key 
     blob_key = files.blobstore.get_blob_key(file_name) 

     return Response('done') 

    def post_to_scribd(self, uploaded_file, fname): 
     errmsg ='' 
     uploaded_file = self.request.files.get('upload_file') 
     fname = uploaded_file.filename.strip() 
     fext = fname[fname.rfind('.')+1:].lower() 
     if (fext not in ALLOWED_EXTENSION): 
      raise Exception('This file type does not allowed to be uploaded\n') 
     if SCRIBD_ENABLED: 
      doc_title = self.request.form.get('title') 
      doc_description = self.request.form.get('description') 
      doc_tags = self.request.form.get('tags') 
      try: 
       document = scribd.api_user.upload(uploaded_file, fname, access='private') 
       #while document.get_conversion_status() != 'DONE': 
       # time.sleep(2) 
       if not doc_title: 
        document.title = fname[:fname.rfind('.')] 
       else: 
        document.title = doc_title 
       if not doc_description: 
        document.description = 'This document was uploaded at ' + str(datetime.datetime.now()) +'\n' 
       else: 
        document.description = doc_description 
       document.tags = doc_tags 
       document.save() 
      except scribd.ResponseError, err: 
       raise Exception('Scribd failed: error code:%d, error message: %s\n' % (err.errno, err.strerror)) 
      except scribd.NotReadyError, err: 
       raise Exception('Scribd failed: error code:%d, error message: %s\n' % (err.errno, err.strerror)) 
      except: 
       raise Exception('something wrong exception')

正如你可以看到它也保存文件到Blob存儲區。但是，如果我上傳大文件（即5MB）我收到

RequestTooLargeError: The request to API call file.Append() was too large. 
Request: docs.upload(access='private', doc_type='pdf', file=('PK\x03\x04\n\x00\x00\x00\x00\x00"\x01\x10=\x00\x00(...)', 'test.pdf'))

我該如何解決它？謝謝！

來源

2011-04-12 minus

你的問題和它的答案幫了我很多，歡呼！ – selurvedu 2015-09-14 16:11:54

你需要讓多個較小的調用文件API，比如像這樣：

with files.open(file_name, 'a') as f: 
    data = uploaded_file.read(65536) 
    while data: 
     f.write(data) 
     data = uploaded_file.read(65536)

注意定期請求App Engine應用有效載荷大小限制爲10MB;如果你想上傳更大的文件，你需要使用常規的blobstore上傳機制。

來源

2011-04-13 01:45:27

使用你的示例代碼，你能想到它爲什麼會給一個AttributeError - 'InMemoryUploadedFile'對象沒有屬性'eof'嗎？（在你的例子的第二行） – oldpal 2011-05-10 20:40:12

@bfox大概是因爲它沒有這個屬性。我會用另一種方式更新我的答案。 – 2011-05-10 21:46:05

@minus您是否想出了一個解決方法？在嘗試將3-4 MB文件上傳到Blob存儲區時，我遇到同樣的問題。 – 2011-06-27 23:14:00

終於我找到了解決辦法。

Nick Johneson的回答發生屬性錯誤，因爲uploaded_file被視爲字符串。字符串沒有read（）方法。

原因字符串沒有方法read（），我拼接文件字符串，並寫它就像他寫。

class UploadRankingHandler(webapp.RequestHandler): 
    def post(self): 
    fish_image_file = self.request.get('file') 

    file_name = files.blobstore.create(mime_type='image/png', _blobinfo_uploaded_filename="testfilename.png") 

    file_str_list = splitCount(fish_image_file,65520) 

    with files.open(file_name, 'a') as f: 
     for line in file_str_list: 
     f.write(line)

你可以檢查splitCount（）。這裏

http://www.bdhwan.com/entry/gaewritebigfile

來源

2011-08-18 11:31:45

如何使用實驗性API將大文件寫入Blobstore？

回答

相關問題