Python不寫頭文件

Question

我用這個基於第二列「ParentID」的python程序分裂出一個非常大的csv。由於大量文件和每個進程限制，我最近更新爲「a」而不是「w」。這樣做我的標題是每次寫入，而不是每個文件中的第一次。

我更新，增加了「write_header = true」和「write_header =假」，但現在它只是寫在第一個文件頭......我有29,000文件

#!/usr/bin/env python3 
import binascii 
import csv 
import os.path 
import sys 
from tkinter.filedialog import askopenfilename, askdirectory 
from tkinter.simpledialog import askinteger 

def split_csv_file(f, dst_dir, keyfunc): 
    csv_reader = csv.reader(f) 
    header = next(csv_reader) 
    write_header = True 
    csv_writers = {} 
    for row in csv_reader: 
     k = keyfunc(row) 
     with open(os.path.join(dst_dir, k), mode='a', newline='') as output: 
      writer = csv.writer(output) 
      while write_header: 
       writer.writerow(header) 
       write_header = False 
      csv_writers[k] = writer 
      csv_writers[k].writerow(row[0:1]) 

def get_args_from_cli(): 
    input_filename = sys.argv[1] 
    column = int(sys.argv[2]) 
    dst_dir = sys.argv[3] 
    return (input_filename, column, dst_dir) 

def get_args_from_gui(): 
    input_filename = askopenfilename(
     filetypes=(('TXT','.txt'),('CSV', '.csv')), 
     title='Select CSV Input File') 
    column = askinteger('Choose Table Column', 'Table column') 
    dst_dir = askdirectory(title='Select Destination Directory') 
    return (input_filename, column, dst_dir) 

if __name__ == '__main__': 
    if len(sys.argv) == 1: 
     input_filename, column, dst_dir = get_args_from_gui() 
    elif len(sys.argv) == 4: 
     input_filename, column, dst_dir = get_args_from_cli() 
    else: 
     raise Exception("Invalid number of arguments") 
    with open(input_filename, mode='r', newline='') as f: 
     split_csv_file(f, dst_dir, lambda r: r[column-1]+'.txt') 
     # if the column has funky values resulting in invalid filenames 
     # replace the line from above with: 
     # split_csv_file(f, dst_dir, lambda r: binascii.b2a_hex(r[column-1].encode('utf-8')).decode('utf-8')+'.csv') 

下面是一個例子的文件被分割..

<option value=''>Choose SubGroup</option>, ParentID 
<option value='/1990-Accord-DX-Glass-s/37918.htm'>Glass</option>,Accord1990DX422F22A1BodyHardwareBackGlass 
<option value='/1990-Accord-DX-Glass-s/37919.htm'>Glass</option>,Accord1990DX422F22A1BodyHardwareBackGlass 
<option value='/1990-Accord-DX-Reveal-Moldings-s/69090.htm'>Reveal Moldings</option>,Accord1990DX422F22A1BodyHardwareBackGlass 
<option value='/1990-Accord-DX-Reveal-Moldings-s/69091.htm'>Reveal Moldings</option>,Accord1990DX422F22A1BodyHardwareBackGlass 
<option value='/1990-Accord-DX-Center-s/10331.htm'>Center</option>,Accord1990DX422F22A1BodyHardwareConsole 
<option value='/1990-Accord-DX-Cowl-s/16006.htm'>Cowl</option>,Accord1990DX422F22A1BodyHardwareCowl 
<option value='/1990-Accord-DX-Exterior-Trim-s/26889.htm'>Exterior Trim</option>,Accord1990DX422F22A1BodyHardwareFender 
<option value='/1990-Accord-DX-Exterior-Trim-s/26890.htm'>Exterior Trim</option>,Accord1990DX422F22A1BodyHardwareFender 

我怎樣才能得到頭每寫輸出文件只有一次？

Answer 1

您在第一次編寫標題時將write_header設置爲false。因此，只有您打開的第一個文件纔會獲得該標題。

跟蹤哪些文件頭中的一組設置：

def split_csv_file(f, dst_dir, keyfunc): 
    csv_reader = csv.reader(f) 
    header = next(csv_reader) 
    header_written = set() 
    for row in csv_reader: 
     k = keyfunc(row) 
     with open(os.path.join(dst_dir, k), mode='a', newline='') as output: 
      writer = csv.writer(output) 
      if k not in header_written: 
       writer.writerow(header) 
       header_written.add(k) 
     writer.writerow(row[0:1]) 

你可能想調查使您的文件，通過追蹤你最後一次寫信給一個和關閉那些你沒有寫，打開更長最長。這需要一個自定義的類，當你通過鍵請求它們時，它會透明地跟蹤文件，而不是放在答案中。