如何使用正則表達式作爲分隔符導入帶有額外逗號的熊貓csv文件？

Question

CSV文件發給我/我不能重新界定列

239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00 

我更換了串字母涵蓋敏感信息，但問題是顯而易見的。

這是我的csv中的「問題行」示例。

col1: 239845723 
col2: 28374 
col3: 2384234 
col4: AEVNE EFU 5 GN OR WNV 
col5: Owinv Vnwo Badvw 5 VIN 
col6: Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.). 
col7: 2011-07-13 00:00:00 
col8: 2011-07-13 00:00:00 

正如你所看到的，列6就是因爲有字符串中的逗號，導致大熊貓界定，並錯誤地創建新列出現問題：應該如下分成8列。我怎麼解決這個問題？我認爲正則表達式可能會有所幫助，也許在下面的設置。任何幫助表示讚賞！

csvfile = open(filetrace) 
    reader = csv.reader(csvfile) 
    new_list=[] 
    for line in reader: 
     for i in line: 
      #not sure 

Answer 1

因此，在不知道文件或數據的細節，我可以提供一個正則表達式的解決方案，可以，如果該數據是一致的（並具有周期在列6月底）工作。我們可以在不使用csv模塊和只使用正則表達式模塊的情況下做到這一點。

import re 

# make the regex pattern here 
pattern = r"([\d\.]*),([\d\.]*),([\d\.]*),([^,]*),([^,]*),(.*\.?),([\d\-\s:]*),([\d\-\s:]*)" 

# open the file with 'with' so you don't have to worry about closing it 
with open(filetrace) as f: 
    for line in f: # iterate through the lines 
     values = re.findall(pattern, line)[0] # re.findall returns a list 
               # literal of a tuple 
     # record the values somewhere 

values這裏是每一個你在原來的CSV有列包含值8元組，只是使用/存儲它們，不過你想要的。

祝你好運！

Answer 2

由於您確切知道需要多少列，並且只有一個有問題的列，因此我們可以將前幾個從左側分開，然後從右側分割。換句話說，你不需要regex

讀文件到一個字符串

csvfile = open(filetrace).read() 

讓pd.Series

s = pd.Series(csvfile.split('\n')) 

拆分s它限制在5次分裂，這應該是6列

df = s.str.split(',', 5, expand=True) 

現在拆分右側limi泰德2個分裂

df = df.iloc[:, :-1].join(df.iloc[-1].str.rsplit(',', 2, expand=True)) 

---

另一種方式從s

left = s.str.split(',', 5) 
right = left.str[-1].str.rsplit(',', 2) 

df = pd.DataFrame(left.str[:-1].add(right).tolist()) 

---

我跑了這一點，並採取了轉開始，使其更容易在屏幕上閱讀

df.T 



                0 
0           239845723 
1            28374 
2           2384234 
3        AEVNE EFU 5 GN OR WNV 
4        Owinv Vnwo Badvw 5 VIN 
5 Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd b... 
6        2011-07-13 00:00:00 
7        2011-07-13 00:00:00 

Answer 3

是去正則表達式，閱讀帶分隔符的csv'，'，您可以提取最後兩個日期並將其存儲在列表中。然後用''填入日期，然後加入你想要的列並刪除剩下的部分。例如

如果你有一個CSV文件：

 
239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00 
239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE.).,2011-07-13 00:00:00,2011-07-13 00:00:00 
239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00 

然後

df = pd.read_csv('good.txt',delimiter=',',header=None) 
# Get the Dates from all the DataFrame 
dates = [[item] for i in df.values for item in i if '2011-' in str(item)] 
# Merge two Dates for each column 
dates = pd.DataFrame([x+y for x,y in zip(dates[0::2], dates[1::2])]) 
# Remove the dates present 
df = df.replace({'2011-': np.nan}, regex=True).replace(np.nan,'') 

#Get the columns you want to merge 
cols = df.columns[4:] 
# Merge the columns 
df[4] = df[cols].astype(str).apply(lambda x: ','.join(x), axis=1) 
df[4] = df[4].replace('\,+$', '',regex=True) 
#Drop the Columns 
df = df.drop(df.columns[5:],axis=1) 
#Concat the dates 
df = pd.concat([df,dates],axis=1) 

輸出：打印（DF）

 
      0  1  2      3 \ 
0 239845723 28374 2384234 AEVNE EFU 5 GN OR WNV 
1 239845723 28374 2384234 AEVNE EFU 5 GN OR WNV 
2 239845723 28374 2384234 AEVNE EFU 5 GN OR WNV 

                4     0 \ 
0 Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera ... 2011-07-13 00:00:00 
1 Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera ... 2011-07-13 00:00:00 
2 Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse... 2011-07-13 00:00:00 

        1 
0 2011-07-13 00:00:00 
1 2011-07-13 00:00:00 
2 2011-07-13 00:00:00 

輸出繼電器的第四列：

 
['Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).', 

'Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE.).', 

'Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).'] 

如果你想改變列索引

df.columns = [i for i in range(df.shape[1])] 

希望它可以幫助