Scrapy：下載svg圖像

-1

我需要從這些網站上刮取一些網站和圖像。當圖像是* .jpg我沒有任何問題，但這些網站也有* .svg圖像，我需要這些。Scrapy：下載svg圖像

有沒有人做過這個？

這裏是shell輸出與錯誤：

2013-01-18 14:44:10-0600 [crawler] DEBUG: Image (downloaded): Downloaded image from <GET http://page/image.svg> referred in <None> 
2013-01-18 14:44:10-0600 [crawler] Unhandled Error 

Traceback (most recent call last): 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 576, in _runCallbacks 
     current.result = callback(current.result, *args, **kw) 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 381, in callback 
     self._startRunCallbacks(result) 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 489, in _startRunCallbacks 
     self._runCallbacks() 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 576, in _runCallbacks 
     current.result = callback(current.result, *args, **kw) 
    --- <exception caught here> --- 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/Scrapy-0.16.3-py2.7.egg/scrapy/contrib/pipeline/images.py", line 199, in media_downloaded 
     checksum = self.image_downloaded(response, request, info) 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/Scrapy-0.16.3-py2.7.egg/scrapy/contrib/pipeline/images.py", line 252, in image_downloaded 
     for key, image, buf in self.get_images(response, request, info): 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/Scrapy-0.16.3-py2.7.egg/scrapy/contrib/pipeline/images.py", line 261, in get_images 
     orig_image = Image.open(StringIO(response.body)) 
File "/virtualenvs/asd/local/lib/python2.7/site-packages/PIL/Image.py", line 1980, in open 
     raise IOError("cannot identify image file") 
    exceptions.IOError: cannot identify image file

謝謝！（對不起，我的英語）

來源

2013-01-18 Fabian Andres Garcia Tenorio

如果幫助別人，我能解決這個如下

它item.py，這些屬性添加到該對象：

 body = Field() 
     url = Field()

在蜘蛛（高清解析（））內，添加以下代碼：

import urllib2 

(...) 

    #select each img url 
    relative_urls = info.select('tr/td/a[@class="image"]/img/@src').extract() 

    for relative_url in relative_urls: 
     #static url 
     relative_url = relative_url.split("svg")[0][2:-1]+".svg" 
     relative_url = ''.join(relative_url.split("/thumb")).strip() 

     relative_url = "http://"+relative_url 

     asd = urllib2.urlopen(relative_url) 
     data = asd.read() 
     with open("%s/%s" % ('/home/user/virtualenvs', img.svg), "wb") as code: 
      code.write(data)

它爲我工作

（顯然可以分開蜘蛛和管道之間的代碼）

來源

2013-01-23 14:05:53

Scrapy：下載svg圖像

回答

相關問題