2012-03-07 45 views
4

我試圖使用Django流水線-1.1.27與s3boto壓縮和過濾靜態文件,然後將它們上傳到S3桶。如果我只是使用:Django的管道和s3boto存儲似乎並不共同努力

PIPELINE_STORAGE = 'pipeline.storage.PipelineFinderStorage' 

然後它工作,我得到一個靜態文件夾與我配置好的版本化文件。當我切換到

PIPELINE_STORAGE = 'storages.backends.s3boto.S3BotoStorage' 

我得到

Traceback (most recent call last): 
    File "manage.py", line 15, in <module> 
    execute_manager(settings) 
    File "/my/virtual/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager 
    utility.execute() 
    File "/my/virtual/env/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute 
    self.fetch_command(subcommand).run_from_argv(self.argv) 
    File "/my/virtual/env/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv 
    self.execute(*args, **options.__dict__) 
    File "/my/virtual/env/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute 
    output = self.handle(*args, **options) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/management/commands/synccompress.py", line 39, in handle 
    packager.pack_stylesheets(package, sync=sync, force=force) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/packager.py", line 52, in pack_stylesheets 
    **kwargs) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/packager.py", line 60, in pack 
    package['output'], package['paths']) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/versioning/__init__.py", line 45, in need_update 
    version = self.version(paths) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/versioning/__init__.py", line 20, in version 
    return getattr(self.versioner, 'version')(paths) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/versioning/hash/__init__.py", line 37, in version 
    buf = self.concatenate(paths) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/versioning/hash/__init__.py", line 27, in concatenate 
    return '\n'.join([self.read_file(path) for path in paths]) 
    File "/my/virtual/env/lib/python2.7/site-packages/pipeline/versioning/hash/__init__.py", line 31, in read_file 
    file = storage.open(path, 'rb') 
    File "/my/virtual/env/lib/python2.7/site-packages/django/core/files/storage.py", line 33, in open 
    file = self._open(name, mode) 
    File "/my/virtual/env/lib/python2.7/site-packages/storages/backends/s3boto.py", line 177, in _open 
    raise IOError('File does not exist: %s' % name) 
IOError: File does not exist: css/style.css 

這是我文件之一。那麼,爲什麼管道不再想做的過濾器/串連/壓縮步驟,當我切換到s3boto存儲?

這可能是因爲我在做什麼。這裏是其他配置的情況下,它可以幫助:

INSTALLED_APPS = (
    ... 
    'pipeline', 
    'storages', 
) 

STATICFILES_FINDERS = (
    'pipeline.finders.PipelineFinder', 
    'django.contrib.staticfiles.finders.FileSystemFinder', 
    'django.contrib.staticfiles.finders.AppDirectoriesFinder', 
) 

STATIC_ROOT = "/some/path/outside/django_project/deploy_static" 
STATICFILES_DIRS =() # All statics in this site are in apps 

STATICFILES_STORAGE = 'pipeline.storage.PipelineStorage' 
PIPELINE_STORAGE = 'storages.backends.s3boto.S3BotoStorage' 

PIPELINE = True 
PIPELINE_AUTO = True 
PIPELINE_VERSION = True 
PIPELINE_VERSION_PLACEHOLDER = 'VERSION' 
PIPELINE_VERSIONING = 'pipeline.versioning.hash.SHA1Versioning' 

PIPELINE_CSS = { 
    'standard': { 
     'source_filenames': (
      'css/style.css', 
      ... 
     ), 
     'output_filename': 'css/all-VERSION.css', 
     'extra_context': { 
      'media': 'screen,projection', 
     }, 
    } 
} 

我的網站是在Django 1.3.1。

我跑的命令是:

python manage.py synccompress --force 

的AWS creds也在設置,但這是沒有實際意義,因爲它甚至沒有得到這一點。

UPDATE加滿堆和評論

UPDATE在圖書館筆者的請求所請求的設置,我試過升級到最新的測試版。從目前觀察:

  1. 我不知道如何版本的壓縮文件現在
  2. collectstatic給我留下的壓縮文件原件
  3. 仍然django-得到同樣的錯誤管道時博託存儲配置:它要送我文件到S3,但我甚至無法看到它的上演我的資產。沒有東西放在STATIC_ROOT中。

UPDATE我已經創建了一個用於取景器的存儲空間,然後用S3Boto打破了簡單的項目。我把它推到了github上,並且包含了一個堆棧跟蹤的捕獲。

https://github.com/estebistec/simple_pipeline https://raw.github.com/estebistec/simple_pipeline/master/STACKTRACE

我會欣喜若狂,如果我能告訴我在做一些非常愚蠢的,這應該都只是工作。

+0

什麼是你設置爲''STATIC_ROOT'',''STATICFILES_DIRS''和''PIPELINE_ROOT''?還有完整的回溯將是有用的! – cyberdelia 2012-03-07 21:05:41

+0

添加完整堆棧,STATIC_ROOT和STATICFILES_DIRS。在1.1.27文檔中,* only *提及PIPELINE_ROOT是在絕對路徑相關的信息框中,我沒有用它來定位源文件。 – StevenC 2012-03-07 21:17:44

+0

那麼我的空STATICFILES_DIRS會把它全部扔掉?不管文檔意味着它應該被使用,是否需要PIPELINE_ROOT? – StevenC 2012-03-07 21:20:05

回答

5

django-pipeline 1.1.x有點啞你應該如何使用staticfiles時,喜歡在一個地方的一切。 我建議你嘗試django-pipeline 1.2與最新django-staticfilesdjango 1.4

使用這樣的風俗:

STATICFILES_STORAGE = 'your.app.S3PipelineStorage' 

的代碼看起來是這樣的:

from staticfiles.storage import CachedFilesMixin 

from pipeline.storage import PipelineMixin 

from storages.backends.s3boto import S3BotoStorage 


class S3PipelineStorage(PipelineMixin, CachedFilesMixin, S3BotoStorage): 
    pass 

你可以找到如何解決您的應用程序,但仍有與除非你編譯後的文件中的錯誤使用版本1.2C1:https://gist.github.com/1999564

+0

對不起,我打算在管道查找器後面加上橢圓。我不*試圖*讓我很難給予幫助:/ – StevenC 2012-03-07 21:19:26

+0

哦哇,我錯過了文檔中的staticfiles依賴關係嗎?我很快從1.2 beta版中墮落,因爲我可以看到它使用的不是django 1.3中的東西。我不知道django-staticfiles仍然在django之外得到增強:/ – StevenC 2012-03-07 21:35:51

+0

文檔總是需要改進,不要猶豫,打開github問題的文檔或其他。 – cyberdelia 2012-03-07 21:38:43

0

作爲補充答案,您可以在壓縮時使用GZIP:

from django.contrib.staticfiles.storage import CachedFilesMixin 

from pipeline.storage import PipelineMixin 

from storages.backends.s3boto import S3BotoStorage 


class S3PipelineStorage(PipelineMixin, CachedFilesMixin, S3BotoStorage): 
    def __init__(self, *args, **kwargs): 
     self.gzip = True 
     super(S3PipelineStorage, self).__init__(*args, **kwargs) 

使用設置如下:

COMPRESS_STORAGE = STATICFILES_STORAGE = 'my.apps.main.S3PipelineStorage' 
2

我只是經歷了一個Django 1.6項目同樣的錯誤與django-pipeline==1.3.23,並將該溶液簡單地移除PIPELINE_STORAGE設置。

0

不知道這似乎適用於其他人。我跟着上述解決方案,並不斷收到以下錯誤:

Traceback (most recent call last): 
    File "manage.py", line 24, in <module> 
    execute_from_command_line(sys.argv) 
    File "python3.4/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line 
    utility.execute() 
    File "python3.4/site-packages/django/core/management/__init__.py", line 330, in execute 
    self.fetch_command(subcommand).run_from_argv(self.argv) 
    File "python3.4/site-packages/django/core/management/base.py", line 390, in run_from_argv 
    self.execute(*args, **cmd_options) 
    File "python3.4/site-packages/django/core/management/base.py", line 441, in execute 
    output = self.handle(*args, **options) 
    File "python3.4/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 168, in handle 
    collected = self.collect() 
    File "python3.4/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 114, in collect 
    for original_path, processed_path, processed in processor: 
    File "python3.4/site-packages/pipeline/storage.py", line 26, in post_process 
    packager.pack_stylesheets(package) 
    File "python3.4/site-packages/pipeline/packager.py", line 96, in pack_stylesheets 
    variant=package.variant, **kwargs) 
    File "python3.4/site-packages/pipeline/packager.py", line 105, in pack 
    paths = self.compile(package.paths, force=True) 
    File "python3.4/site-packages/pipeline/packager.py", line 99, in compile 
    return self.compiler.compile(paths, force=force) 
    File "python3.4/site-packages/pipeline/compilers/__init__.py", line 56, in compile 
    return list(executor.map(_compile, paths)) 
    File "/usr/local/lib/python3.4/concurrent/futures/_base.py", line 549, in result_iterator 
    yield future.result() 
    File "/usr/local/lib/python3.4/concurrent/futures/_base.py", line 402, in result 
    return self.__get_result() 
    File "/usr/local/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result 
    raise self._exception 
    File "/usr/local/lib/python3.4/concurrent/futures/thread.py", line 54, in run 
    result = self.fn(*self.args, **self.kwargs) 
    File "python3.4/site-packages/pipeline/compilers/__init__.py", line 42, in _compile 
    outdated = compiler.is_outdated(input_path, output_path) 
    File "python3.4/site-packages/pipeline/compilers/__init__.py", line 85, in is_outdated 
    return self.storage.modified_time(infile) > self.storage.modified_time(outfile) 
    File "python3.4/site-packages/storages/backends/s3boto.py", line 480, in modified_time 
    return parse_ts(entry.last_modified) 
AttributeError: 'NoneType' object has no attribute 'last_modified' 

直到我碰到this solution,我就開始找我有什麼工作來了。下面是我最終使用保存在本地文件以及在S3這讓我通過了所有的錯誤存儲:

from django.contrib.staticfiles.storage import ManifestFilesMixin 
from django.core.files.storage import get_storage_class 
from pipeline.storage import PipelineMixin 
from storages.backends.s3boto import S3BotoStorage 


class StaticStorage(PipelineMixin, ManifestFilesMixin, S3BotoStorage): 
    """Custom storage for static content.""" 

    def __init__(self, *args, **kwargs): 
     super(StaticStorage, self).__init__(*args, **kwargs) 
     self.local_storage = get_storage_class(
      'django.contrib.staticfiles.storage.StaticFilesStorage')() 

    def save(self, name, content): 
     name = super(StaticStorage, self).save(name, content) 
     self.local_storage._save(name, content) 
     return name 
+0

所以這最初是針對較舊版本的管道。不幸的是,我似乎沒有能夠編輯問題標題來放入一個版本,並明確說明:/ – StevenC 2015-05-16 22:49:53

2

有與早期影響類似的錯誤信息和當前版本的另一個問題(1.5.4 )的django-pipeline

錯誤消息是IOError: File does not exist,它發生在s3boto.py.open()packager.pack_stylesheets()。如果你使用任何編譯器(Compass,Sass,Less等),你可能會遇到問題。我懷疑它也會影響JS編譯器,但我還沒有確認。

簡而言之,編譯器會在本地靜態存儲器中生成輸出文件,並且接下來的步驟compress正試圖在s3存儲中查找輸出。

如果它影響到你,你可能想看看https://github.com/cyberdelia/django-pipeline/issues/473。有兩個拉請求(補丁),一個由skirsdeda製作,另一個由thomasyip(我)製作。兩者都可以解決你的問題。如果您希望將編譯的(但是,預先壓縮的)文件複製到s3並且可用於應用程序,那麼您將採用thomasyip(me)的補丁。

這裏是問題完全回溯:

Traceback (most recent call last): 
    File "apps/manage.py", line 16, in <module> 
    execute_from_command_line(sys.argv) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line 
    utility.execute() 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute 
    self.fetch_command(subcommand).run_from_argv(self.argv) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv 
    self.execute(*args, **options.__dict__) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute 
    output = self.handle(*args, **options) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/management/base.py", line 533, in handle 
    return self.handle_noargs(**options) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 171, in handle_noargs 
    collected = self.collect() 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 117, in collect 
    for original_path, processed_path, processed in processor: 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/storage.py", line 26, in post_process 
    packager.pack_stylesheets(package) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/packager.py", line 96, in pack_stylesheets 
    variant=package.variant, **kwargs) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/packager.py", line 106, in pack 
    content = compress(paths, **kwargs) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 73, in compress_css 
    css = self.concatenate_and_rewrite(paths, output_filename, variant) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 137, in concatenate_and_rewrite 
    content = self.read_text(path) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 220, in read_text 
    content = self.read_bytes(path) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 214, in read_bytes 
    file = staticfiles_storage.open(path) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/django/core/files/storage.py", line 35, in open 
    return self._open(name, mode) 
    File "/Users/thomas/Dev/Project/main-server/venv/lib/python2.7/site-packages/storages/backends/s3boto.py", line 366, in _open 
    raise IOError('File does not exist: %s' % name) 
IOError: File does not exist: sheets/sass/sheets.css 
相關問題