2012-08-09 107 views
28

我已經從[python網站]複製了此腳本[1]這是另一個問題,但是現在編碼的問題:python csv unicode'ascii'編解碼器無法編碼字符u' xf6'在位置1:序號不在範圍內(128)

import sqlite3 
import csv 
import codecs 
import cStringIO 
import sys 

class UTF8Recoder: 
    """ 
    Iterator that reads an encoded stream and reencodes the input to UTF-8 
    """ 
    def __init__(self, f, encoding): 
     self.reader = codecs.getreader(encoding)(f) 

    def __iter__(self): 
     return self 

    def next(self): 
     return self.reader.next().encode("utf-8") 

class UnicodeReader: 
    """ 
    A CSV reader which will iterate over lines in the CSV file "f", 
    which is encoded in the given encoding. 
    """ 

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): 
     f = UTF8Recoder(f, encoding) 
     self.reader = csv.reader(f, dialect=dialect, **kwds) 

    def next(self): 
     row = self.reader.next() 
     return [unicode(s, "utf-8") for s in row] 

    def __iter__(self): 
     return self 

class UnicodeWriter: 
    """ 
    A CSV writer which will write rows to CSV file "f", 
    which is encoded in the given encoding. 
    """ 

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): 
     # Redirect output to a queue 
     self.queue = cStringIO.StringIO() 
     self.writer = csv.writer(self.queue, dialect=dialect, **kwds) 
     self.stream = f 
     self.encoder = codecs.getincrementalencoder(encoding)() 

    def writerow(self, row): 
     self.writer.writerow([s.encode("utf-8") for s in row]) 
     # Fetch UTF-8 output from the queue ... 
     data = self.queue.getvalue() 
     data = data.decode("utf-8") 
     # ... and reencode it into the target encoding 
     data = self.encoder.encode(data) 
     # write to the target stream 
     self.stream.write(data) 
     # empty queue 
     self.queue.truncate(0) 

    def writerows(self, rows): 
     for row in rows: 
      self.writerow(row) 

這次問題的編碼,當我跑這一點,給了我這個錯誤:

Traceback (most recent call last): 
    File "makeCSV.py", line 87, in <module> 
    uW.writerow(d) 
    File "makeCSV.py", line 54, in writerow 
    self.writer.writerow([s.encode("utf-8") for s in row]) 
AttributeError: 'int' object has no attribute 'encode' 

然後我轉換所有整數到字符串,但這個時候,我得到這個錯誤:

Traceback (most recent call last): 
    File "makeCSV.py", line 87, in <module> 
    uW.writerow(d) 
    File "makeCSV.py", line 54, in writerow 
    self.writer.writerow([str(s).encode("utf-8") for s in row]) 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 1: ordinal not in range(128) 

我已經實現了上面處理unicode字符,但它給了我這樣的錯誤。有什麼問題以及如何解決它?

回答

64

Then I converted all integers to string,

您轉換均爲整數字符串字節串。對於字符串,這將使用默認字符編碼,它恰好是ASCII,並且當您使用非ASCII字符時會失敗。你想要unicode而不是str

self.writer.writerow([unicode(s).encode("utf-8") for s in row]) 

在調用該方法之前將所有內容轉換爲unicode可能會更好。該類專門用於解析Unicode字符串。它不是爲了支持其他數據類型而設計的。

相關問題