Scrapy IO錯誤：[錯誤22]無效的模式（「WB」）或文件名

Question

import scrapy 
from scrapy.spiders import CrawlSpider, Rule 
from scrapy.linkextractors import LinkExtractor 
from scrapy.selector import HtmlXPathSelector 
import os 
from Erowid.items import ErowidItem 
import codecs 

class ExperiencesSpider(CrawlSpider): 
    name = "experiences" 
    allowed_domains = ["www.erowid.org"] 
    start_urls = ['https://www.erowid.org/experiences/exp_list.shtml'] 
    rules = [ 
     Rule(LinkExtractor(allow =('subs/exp_[a-zA-Z]+.shtml')), follow = True), 
     Rule(LinkExtractor(allow=r'/experiences/exp\.php\?ID=\d+$'),callback='parse_item', follow = True) 
    ] 
    def parse_item(self, response): 
     selectors = response.css('div') 
     for selector in selectors: 
      experience = ErowidItem() 
      experience['Author'] = selector.xpath('//div[@class="author"]/a/text()').extract() 
      experience['Title'] = selector.xpath('//div[@class="title"]/text()').extract() 
      experience['Substance'] = selector.xpath('//div[@class="substance"]/text()').extract() 
      experience['Text'] = selector.xpath("//div[@class = 'report-text-surround']/text()").extract() 
      experience['Title'] = str(experience['Title']).replace('\\' , "") 
      experience['Title'] = str(experience['Title']).replace('?' , "") 

      directory = os.path.join('Erowid/archive/',experience['Substance'][0].strip().lower()) 
      filename = os.path.join(directory,experience['Title'][0]+'.txt') 

      if not os.path.exists(directory): 
       os.makedirs(directory) 

      with codecs.open(filename, encoding = 'utf-8', mode= 'wb') as fid: 
       for symbols in experience['Text']: 
        fid.write(symbols) 
     yield experience 

我試圖刮Erowid，到目前爲止，該代碼是由這樣我將創建一個目錄，寫主要基於文件經驗的名稱和文本。 （根據經驗，我指的是我從Erowid處取消的信息。）

問題是，某些經驗在其名稱中包含不能寫爲文件的字符，因爲它們包含的保留字符可以將不會在windows filenames中使用。

我試圖消除所有這些保留的字符，這是

• The following reserved characters:

  • < (less than)
  • > (greater than)
  • : (colon)
  • " (double quote)
  • / (forward slash)
  • \ (backslash)
  • | (vertical bar or pipe)
  • ? (question mark)
  • * (asterisk)

的我試圖做一些與線

experience['Title'] = str(experience['Title']).replace('\\' , "") 
experience['Title'] = str(experience['Title']).replace('?' , "") 

（大多數錯誤來自「\」和「？」）但我仍然收到錯誤

IOError: [Errno 22] invalid mode ('wb') or filename: u'Erowid/archive/syrian rue\\Meditative Help?.txt' 

或其他一些我知道的文件名是wro因爲不應該有問號或反斜槓。

我在做什麼錯誤觸發這些錯誤？

Answer 1

您未能替換特殊字符。試試這個：

filename = os.path.join(directory,experience['Title'][0]+'.txt') 
filename = filename.replace('\\' , "").replace('?' , "") 

UPDATE

你只是想指定一個合法的文件名。所以我想出了一個這樣的想法。

directory = os.path.join('Erowid/archive/',experience['Substance'][0].strip().lower()) 
filename = experience['Substance']+experience['Title'][0]+'.txt' 
filename = "".join([i for i in filename if i in string.ascii_letters]) 
#only use ascii letters as file name 
filename = os.path.join(directory, filename) 

string.ascii_letters

The concatenation of the ascii_lowercase and ascii_uppercase constants described below. This value is not locale-dependent.