使用python讀取CSV文件時的編碼問題

Question

嘗試使用python讀取CSV文件時，遇到了路障。

UPDATE： 如果你只想跳過字符或錯誤，您可以打開該文件是這樣的：

with open(os.path.join(directory, file), 'r', encoding="utf-8", errors="ignore") as data_file: 

到目前爲止，我都試過了。

for directory, subdirectories, files in os.walk(root_dir): 
    for file in files: 
     with open(os.path.join(directory, file), 'r') as data_file: 
      reader = csv.reader(data_file) 
      for row in reader: 
       print (row) 

我得到的錯誤是：

UnicodeEncodeError: 'charmap' codec can't encode characters in position 224-225: character maps to <undefined> 

我已經試過

with open(os.path.join(directory, file), 'r', encoding="UTF-8") as data_file: 

錯誤：

UnicodeEncodeError: 'charmap' codec can't encode character '\u2026' in position 223: character maps to <undefined> 

現在，如果我只是打印DATA_FILE它說，他們是cp1252編碼，但如果我嘗試

with open(os.path.join(directory, file), 'r', encoding="cp1252") as data_file: 

我得到的錯誤是：

UnicodeEncodeError: 'charmap' codec can't encode characters in position 224-225: character maps to <undefined> 

我也試過推薦的包。

我得到的錯誤是：

UnicodeEncodeError: 'charmap' codec can't encode characters in position 224-225: character maps to <undefined> 

我試圖解析該生產線是：

2015-11-28 22:23:58,670805374291832832,479174464,"MarkCrawford15","RT @WhatTheFFacts: The tallest man in the world was Robert Pershing Wadlow of Alton, Illinois. He was slighty over 8 feet 11 inches tall.","None 

任何想法或幫助表示讚賞。

Answer 1

我會使用csvkit，它使用自適應編碼和解碼檢測。例如

import csvkit 
reader = csvkit.reader(data_file) 

如chat-溶液是 -

for directory, subdirectories, files in os.walk(root_dir): 
    for file in files: 
     with open(os.path.join(directory, file), 'r', encoding="utf-8") as data_file: 
      reader = csv.reader(data_file) 
      for row in reader: 
       data = [i.encode('ascii', 'ignore').decode('ascii') for i in row] 
       print (data)