UnicodeDecodeError：'utf-8'編碼解碼器無法解碼位置1中的字節0x8b：無效的起始字節，同時讀取pandas中的csv文件

Question

我知道類似的問題已經被問到，我已經看過所有這些問題，幫幫我。我正在使用OSX 10.11 El Capitan，python3.6。，虛擬環境，也沒有嘗試過。我正在使用jupyter筆記本和spyder3。

我是新來的蟒蛇，但要知道基本的ML和以下後，以瞭解如何解決Kaggle挑戰：Link to Blog，Link to Data Set

。我是停留在代碼的前幾行 `

import pandas as pd 

destinations = pd.read_csv("destinations.csv") 
test = pd.read_csv("test.csv") 
train = pd.read_csv("train.csv") 

，它是給我的錯誤

UnicodeDecodeError      Traceback (most recent call last) 
<ipython-input-19-a928a98eb1ff> in <module>() 
     1 import pandas as pd 
----> 2 df = pd.read_csv('destinations.csv', compression='infer',date_parser=True, usecols=([0,1,3])) 
     3 df.head() 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision) 
    653      skip_blank_lines=skip_blank_lines) 
    654 
--> 655   return _read(filepath_or_buffer, kwds) 
    656 
    657  parser_f.__name__ = name 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds) 
    403 
    404  # Create the parser. 
--> 405  parser = TextFileReader(filepath_or_buffer, **kwds) 
    406 
    407  if chunksize or iterator: 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self, f, engine, **kwds) 
    762    self.options['has_index_names'] = kwds['has_index_names'] 
    763 
--> 764   self._make_engine(self.engine) 
    765 
    766  def close(self): 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _make_engine(self, engine) 
    983  def _make_engine(self, engine='c'): 
    984   if engine == 'c': 
--> 985    self._engine = CParserWrapper(self.f, **self.options) 
    986   else: 
    987    if engine == 'python': 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self, src, **kwds) 
    1603   kwds['allow_leading_cols'] = self.index_col is not False 
    1604 
-> 1605   self._reader = parsers.TextReader(src, **kwds) 
    1606 
    1607   # XXX 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__ (pandas/_libs/parsers.c:6175)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._get_header (pandas/_libs/parsers.c:9691)() 

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte 

一些答案上stakoverflow認爲，這是因爲它我但是Chrome下載了.csv文件，並且.csv.gz無處可見，並且返回了文件未找到的錯誤。

然後我讀的地方使用encoding='latin1'，但這樣做後，我得到解析器的錯誤：

--------------------------------------------------------------------------- 
ParserError        Traceback (most recent call last) 
<ipython-input-21-f9c451f864a2> in <module>() 
     1 import pandas as pd 
     2 
----> 3 destinations = pd.read_csv("destinations.csv",encoding='latin1') 
     4 test = pd.read_csv("test.csv") 
     5 train = pd.read_csv("train.csv") 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision) 
    653      skip_blank_lines=skip_blank_lines) 
    654 
--> 655   return _read(filepath_or_buffer, kwds) 
    656 
    657  parser_f.__name__ = name 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds) 
    409 
    410  try: 
--> 411   data = parser.read(nrows) 
    412  finally: 
    413   parser.close() 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 
    1003     raise ValueError('skipfooter not supported for iteration') 
    1004 
-> 1005   ret = self._engine.read(nrows) 
    1006 
    1007   if self.options.get('as_recarray'): 

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 
    1746  def read(self, nrows=None): 
    1747   try: 
-> 1748    data = self._reader.read(nrows) 
    1749   except StopIteration: 
    1750    if self._first_chunk: 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)() 

ParserError: Error tokenizing data. C error: Expected 2 fields in line 11, saw 3 

我花了幾個小時來調試這一點，試圖打開原子上的CSV文件（沒有其他的應用程序可能打開它），在線網絡應用程序（一些崩潰），但沒有幫助。我已經嘗試使用已解決問題的其他人的內核，但沒有幫助。

Answer 1

它仍然最有可能的gzip壓縮數據。 gzip的幻數是0x1f 0x8b，這與您得到的UnicodeDecodeError一致。

你可以嘗試解壓縮在飛行中的數據：

with open('destinations.csv', 'rb') as fd: 
    gzip_fd = gzip.GzipFile(fileobj=fd) 
    destinations = pd.read_csv(gzip_fd) 

Answer 2

你可以嘗試使用的編解碼器

import codecs 
with codecs.open("destinations.csv", "r",encoding='utf-8', errors='ignore') as file_dat: 
    destinations = pd.read_csv(file_data))