Python pandas - read_csv Dataframe正在刪除列中的值

Question

首先我知道下面的示例在縮進中關閉，如果有人可以指向正確發送DataFrames的正確方向，我將非常感激。

現在，我有如下一個數據幀：

Ticker_x \t Date \t Close_x \t Ticker_y \t Close_y \t Lead_Lag \t Ticker \t 15M_Trend \t Bollinger_1SD \t Bollinger_2SD \t Bollinger_and_Trend_1SD \t Bollinger_and_Trend_2SD \t 
 
ES M7 \t 6/2/2017 3:29 \t 2433.75 \t VX M7-CF \t 11.6 \t N \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 4:29 \t 2434.75 \t VX M7-CF \t 11.6 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 5:29 \t 2435 \t VX M7-CF \t 11.55 \t N \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 6:29 \t 2434.25 \t VX M7-CF \t 11.57 \t N \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 7:29 \t 2430.25 \t VX M7-CF \t 11.7 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 8:29 \t 2430.75 \t VX M7-CF \t 11.58 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 9:29 \t 2433.25 \t VX M7-CF \t 11.63 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 10:29 \t 2436.75 \t VX M7-CF \t 11.61 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 11:29 \t 2436.75 \t VX M7-CF \t 11.57 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 12:29 \t 2435.75 \t VX M7-CF \t 11.67 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 13:29 \t 2437.75 \t VX M7-CF \t 11.64 \t Short \t \t \t \t \t \t \t 
 
ES M7 \t 6/2/2017 14:29 \t 2437.75 \t VX M7-CF \t 11.63 \t Short \t \t \t \t \t \t \t

我創建從另一個文件的read_csv功能這個數據幀，並且只包括下面列出列。請注意，我從一個文件夾拿起最新的文件與「水珠」：

filenameA = max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv")) 

AggregatedA = pd.read_csv(filenameA, usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD']) 
p2 = r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv" 
AggregatedA.to_csv(path_or_buf = p2) 

，使新文件「blah2」是上面顯示的CSV文件。你會注意到，最右邊的7或8列是空白的。這些不應該是空白的。它們在我從'filenameA'中找到的原始文件中不是空白的。

I.e.他們有一堆'Ticker'，'15M_Trend'和其他欄目的值。由於某種原因，大熊貓沒有選擇這些值，我不知道爲什麼。非常感謝幫助！

編輯：這裏的數據幀的原線路：

Ticker_x \t Date \t Open_x \t High_x \t Low_x \t Close_x \t Volume_x \t Open Interest_x \t Ticker_y \t Open_y \t High_y \t Low_y \t Close_y \t Volume_y \t Open Interest_y \t ES_returns \t VX_returns \t Beta \t Pairs_Spread \t zscore \t Pairs_Spread_Mean \t Pairs_Spread_sdev \t ES_percent_change \t ES_difference \t VX_percent_change \t ES_CC \t VX_CC \t pairs_spread \t pairs_zscore \t ES_Inverse_price \t ES__Inverse_percent_change \t Inverse_ES_CC \t Inverse_pairs_spread \t Inverse_pairs_zscore \t Lead_Lag \t Ticker \t Open \t High \t Low \t Close \t Volume \t Open Interest \t Index_Num \t Rolling_OLS_Coefficient \t 15M_Long \t Upper_Sdev_Value \t Lower_Sdev_Value \t Intercept \t Middle \t Sdev \t Lower_Sdev_value \t 15M_Trend \t Rolling_mean \t Rolling_std \t Upper_Band \t Lower_Band \t Upper_Band_2 \t Lower_Band_2 \t Bollinger_1SD \t Bollinger_2SD \t Bollinger_and_Trend_1SD \t Bollinger_and_Trend_2SD \t Trend_and_LL 
 
ES M7 \t 6/2/2017 3:29 \t 2433.25 \t 2433.75 \t 2433 \t 2433.75 \t 3419 \t 0 \t VX M7-CF \t 11.55 \t 11.6 \t 11.53 \t 11.6 \t 253 \t 0 \t 0 \t 0.001727116 \t -6.946994692 \t \t \t \t \t 0 \t 0 \t 0.001727116 \t -0.27013895 \t -0.27013895 \t 2514.335138 \t 2.733208694 \t 2138.25 \t 0 \t 0.270990395 \t 2218.835138 \t -1.144384967 \t N \t VX M7-CF \t 11.55 \t 11.6 \t 11.53 \t 11.6 \t 253 \t 0 \t 1868 \t -0.006269114 \t \t 27.09349175 \t \t 23.18140592 \t 23.10868419 \t 3.984807553 \t 19.12387664 \t Short \t 11.7912 \t 0.1573199 \t 11.9485199 \t 11.6338801 \t 12.1058398 \t 11.4765602 \t Long \t N \t N \t N \t N 
 
ES M7 \t 6/2/2017 4:29 \t 2435 \t 2435.5 \t 2434.75 \t 2434.75 \t 2847 \t 0 \t VX M7-CF \t 11.59 \t 11.6 \t 11.55 \t 11.6 \t 118 \t 0 \t 0.000410889 \t 0 \t -6.94464418 \t \t \t \t \t 0.000410889 \t 1 \t 0 \t -0.434541317 \t -0.434541317 \t 2515.307872 \t 2.759879939 \t 2137.25 \t -0.000467672 \t 0.43443517 \t 2217.807872 \t -1.197621083 \t Short \t VX M7-CF \t 11.59 \t 11.6 \t 11.55 \t 11.6 \t 118 \t 0 \t 1869 \t -0.006344733 \t \t 27.26010073 \t \t 23.31610124 \t 23.24250234 \t 4.017598384 \t 19.22490396 \t Short \t 11.7868 \t 0.159558575 \t 11.94635857 \t 11.62724143 \t 12.10591715 \t 11.46768285 \t Long \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 5:29 \t 2436.25 \t 2436.5 \t 2435 \t 2435 \t 5979 \t 0 \t VX M7-CF \t 11.55 \t 11.6 \t 11.5 \t 11.55 \t 716 \t 0 \t 0.00010268 \t -0.004310345 \t -6.945304375 \t \t \t \t \t 0.00010268 \t 0.25 \t -0.004310345 \t -0.364155097 \t -0.364155097 \t 2515.218266 \t 2.75742302 \t 2137 \t -0.000116973 \t 0.364203518 \t 2217.218266 \t -1.228176348 \t N \t VX M7-CF \t 11.55 \t 11.6 \t 11.5 \t 11.55 \t 716 \t 0 \t 1870 \t -0.006422122 \t \t 27.43061644 \t \t 23.45376171 \t 23.37958619 \t 4.051030244 \t 19.32855595 \t Short \t 11.7814 \t 0.162944964 \t 11.94434496 \t 11.61845504 \t 12.10728993 \t 11.45551007 \t Long \t N \t N \t N \t N 
 
ES M7 \t 6/2/2017 6:29 \t 2434.5 \t 2435 \t 2433.75 \t 2434.25 \t 11821 \t 0 \t VX M7-CF \t 11.5 \t 11.65 \t 11.5 \t 11.57 \t 1919 \t 0 \t -0.000308008 \t 0.001731602 \t -6.943255051 \t \t \t \t \t -0.000308008 \t -0.75 \t 0.001731602 \t -0.391364705 \t -0.391364705 \t 2514.583461 \t 2.74001741 \t 2137.75 \t 0.000350959 \t 0.391371805 \t 2218.083461 \t -1.183339232 \t N \t VX M7-CF \t 11.5 \t 11.65 \t 11.5 \t 11.57 \t 1919 \t 0 \t 1871 \t -0.006492588 \t \t 27.58935246 \t \t 23.57945638 \t 23.50433714 \t 4.08501532 \t 19.41932182 \t Short \t 11.7764 \t 0.165551151 \t 11.94195115 \t 11.61084885 \t 12.1075023 \t 11.4452977 \t Long \t N \t N \t N \t N 
 
ES M7 \t 6/2/2017 7:29 \t 2431.25 \t 2431.75 \t 2430 \t 2430.25 \t 16785 \t 0 \t VX M7-CF \t 11.6 \t 11.75 \t 11.58 \t 11.7 \t 3737 \t 0 \t -0.001643217 \t 0.011235955 \t -6.941089619 \t \t \t \t \t -0.001643217 \t -4 \t 0.011235955 \t -0.795190709 \t -0.795190709 \t 2511.460749 \t 2.654396241 \t 2141.75 \t 0.001871126 \t 0.79514286 \t 2222.960749 \t -0.930583022 \t Short \t VX M7-CF \t 11.6 \t 11.75 \t 11.58 \t 11.7 \t 3737 \t 0 \t 1872 \t -0.006552806 \t \t 27.73075716 \t \t 23.68800858 \t 23.61134075 \t 4.119416411 \t 19.49192434 \t Short \t 11.776 \t 0.165714286 \t 11.94171429 \t 11.61028571 \t 12.10742857 \t 11.44457143 \t N \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 8:29 \t 2430.25 \t 2431 \t 2429 \t 2430.75 \t 37511 \t 0 \t VX M7-CF \t 11.7 \t 11.8 \t 11.53 \t 11.58 \t 15635 \t 0 \t 0.00020574 \t -0.01025641 \t -6.944443722 \t \t \t \t \t 0.00020574 \t 0.5 \t -0.01025641 \t -0.801967629 \t -0.801967629 \t 2511.166658 \t 2.646332626 \t 2141.25 \t -0.000233454 \t 0.801808413 \t 2221.666658 \t -0.997646802 \t Short \t VX M7-CF \t 11.7 \t 11.8 \t 11.53 \t 11.58 \t 15635 \t 0 \t 1873 \t -0.006629471 \t \t 27.90272657 \t \t 23.82506069 \t 23.74829141 \t 4.154435156 \t 19.59385626 \t Short \t 11.773 \t 0.167907287 \t 11.94090729 \t 11.60509271 \t 12.10881457 \t 11.43718543 \t Long \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 9:29 \t 2432.25 \t 2433.75 \t 2432.25 \t 2433.25 \t 26047 \t 0 \t VX M7-CF \t 11.55 \t 11.7 \t 11.55 \t 11.63 \t 7148 \t 0 \t 0.001028489 \t 0.004317789 \t -6.902663206 \t \t \t \t \t 0.001028489 \t 2.5 \t 0.004317789 \t -0.50559849 \t -0.50559849 \t 2513.527973 \t 2.711077152 \t 2138.75 \t -0.001167542 \t 0.506119457 \t 2219.027973 \t -1.134391675 \t Short \t VX M7-CF \t 11.55 \t 11.7 \t 11.55 \t 11.63 \t 7148 \t 0 \t 1874 \t -0.006703266 \t \t 28.06957526 \t \t 23.95751485 \t 23.87955587 \t 4.190019391 \t 19.68953648 \t Short \t 11.7706 \t 0.169096157 \t 11.93969616 \t 11.60150384 \t 12.10879231 \t 11.43240769 \t N \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 10:29 \t 2435.75 \t 2437 \t 2435.5 \t 2436.75 \t 31594 \t 0 \t VX M7-CF \t 11.65 \t 11.65 \t 11.55 \t 11.61 \t 5090 \t 0 \t 0.001438405 \t -0.00171969 \t -6.862573345 \t \t \t \t \t 0.001438405 \t 3.5 \t -0.00171969 \t -0.494655304 \t -0.494655304 \t 2516.424477 \t 2.790495933 \t 2135.25 \t -0.00163647 \t 0.495088995 \t 2214.924477 \t -1.347047627 \t Short \t VX M7-CF \t 11.65 \t 11.65 \t 11.55 \t 11.61 \t 5090 \t 0 \t 1875 \t -0.006773515 \t \t 28.23118528 \t \t 24.08370058 \t 24.00506007 \t 4.226125209 \t 19.77893486 \t Short \t 11.7608 \t 0.163729992 \t 11.92452999 \t 11.59707001 \t 12.08825998 \t 11.43334002 \t N \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 11:29 \t 2436 \t 2437 \t 2435.5 \t 2436.75 \t 25538 \t 0 \t VX M7-CF \t 11.65 \t 11.65 \t 11.55 \t 11.57 \t 2278 \t 0 \t 0 \t -0.003445306 \t -6.86522263 \t \t \t \t \t 0 \t 0 \t -0.003445306 \t -0.46654779 \t -0.46654779 \t 2516.180626 \t 2.783809828 \t 2135.25 \t 0 \t 0.466999626 \t 2214.680626 \t -1.359684729 \t Short \t VX M7-CF \t 11.65 \t 11.65 \t 11.55 \t 11.57 \t 2278 \t 0 \t 1876 \t -0.00683482 \t \t 28.37745101 \t \t 24.19388406 \t 24.11480519 \t 4.262645818 \t 19.85215938 \t Short \t 11.7512 \t 0.160466666 \t 11.91166667 \t 11.59073333 \t 12.07213333 \t 11.43026667 \t Long \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 12:29 \t 2437.75 \t 2438.75 \t 2435.5 \t 2435.75 \t 37037 \t 0 \t VX M7-CF \t 11.58 \t 11.69 \t 11.55 \t 11.67 \t 4309 \t 0 \t -0.000410383 \t 0.008643042 \t -6.890566028 \t \t \t \t \t -0.000410383 \t -1 \t 0.008643042 \t -0.503091577 \t -0.503091577 \t 2516.162906 \t 2.783323958 \t 2136.25 \t 0.000468329 \t 0.503595438 \t 2216.662906 \t -1.25695683 \t Short \t VX M7-CF \t 11.58 \t 11.69 \t 11.55 \t 11.67 \t 4309 \t 0 \t 1877 \t -0.006883805 \t \t 28.50205329 \t \t 24.28297379 \t 24.20263978 \t 4.299413516 \t 19.90322626 \t Short \t 11.7436 \t 0.154929345 \t 11.89852935 \t 11.58867065 \t 12.05345869 \t 11.43374131 \t N \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 13:29 \t 2437.75 \t 2438.5 \t 2437 \t 2437.75 \t 26228 \t 0 \t VX M7-CF \t 11.68 \t 11.7 \t 11.55 \t 11.64 \t 8570 \t 0 \t 0.000821102 \t -0.002570694 \t -6.88244494 \t \t \t \t \t 0.000821102 \t 2 \t -0.002570694 \t -0.520801285 \t -0.520801285 \t 2517.861659 \t 2.82990182 \t 2134.25 \t -0.00093622 \t 0.521308056 \t 2214.361659 \t -1.376214577 \t Short \t VX M7-CF \t 11.68 \t 11.7 \t 11.55 \t 11.64 \t 8570 \t 0 \t 1878 \t -0.006929786 \t \t 28.62236572 \t \t 24.3666349 \t 24.2859722 \t 4.336393523 \t 19.94957867 \t Short \t 11.734 \t 0.145728291 \t 11.87972829 \t 11.58827171 \t 12.02545658 \t 11.44254342 \t N \t N \t N \t N \t Short 
 
ES M7 \t 6/2/2017 14:29 \t 2437.75 \t 2437.75 \t 2437.75 \t 2437.75 \t 1 \t 0 \t VX M7-CF \t 11.6 \t 11.7 \t 11.55 \t 11.63 \t 28177 \t 0 \t 0 \t -0.000859107 \t -6.904693941 \t \t \t \t \t 0 \t 0 \t -0.000859107 \t -0.516701901 \t -0.516701901 \t 2518.051591 \t 2.835109521 \t 2134.25 \t 0 \t 0.517232222 \t 2214.551591 \t -1.36637174 \t Short \t VX M7-CF \t 11.6 \t 11.7 \t 11.55 \t 11.63 \t 28177 \t 0 \t 1879 \t -0.006969382 \t \t 28.73142917 \t \t 24.4389793 \t 24.35792539 \t 4.37350378 \t 19.98442161 \t Short \t 11.724 \t 0.13474087 \t 11.85874087 \t 11.58925913 \t 11.99348174 \t 11.45451826 \t N \t N \t N \t N \t Short

編輯：這裏是新的代碼，基於以下建議：

AggregatedA = pd.read_csv(max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv")), usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t') 

AggregatedA.to_csv(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv") 

這給錯誤：

ValueError: Usecols do not match names. 

編輯：我也試過使用delimiter ='，'而不是delimiter ='\ t'，這樣就消除了錯誤。但是，右欄保持空白。

Answer 1

假設輸入是給定的，令人驚訝的是pandas完全解析數據幀，因爲應明確給出分隔符。這樣做，代碼就能正常工作。如果in.txt包含線，你提供的，那麼在運行

In [26]: AggregatedA = pd.read_csv('in.txt', usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t') 

In [25]: AggregatedA.to_csv('out.txt') 

會給一個out.txt具有以下內容：

$ cat out.txt 
,Ticker_x,Date,Close_x,Ticker_y,Close_y,Lead_Lag,Ticker,15M_Trend,Bollinger_1SD,Bollinger_2SD,Bollinger_and_Trend_1SD,Bollinger_and_Trend_2SD 
0,ES M7,6/2/2017 3:29,2433.75,VX M7-CF,11.6,N,VX M7-CF,Short,Long,N,N,N 
1,ES M7,6/2/2017 4:29,2434.75,VX M7-CF,11.6,Short,VX M7-CF,Short,Long,N,N,N 
2,ES M7,6/2/2017 5:29,2435.0,VX M7-CF,11.55,N,VX M7-CF,Short,Long,N,N,N 
3,ES M7,6/2/2017 6:29,2434.25,VX M7-CF,11.57,N,VX M7-CF,Short,Long,N,N,N 
4,ES M7,6/2/2017 7:29,2430.25,VX M7-CF,11.7,Short,VX M7-CF,Short,N,N,N,N 
5,ES M7,6/2/2017 8:29,2430.75,VX M7-CF,11.58,Short,VX M7-CF,Short,Long,N,N,N 
6,ES M7,6/2/2017 9:29,2433.25,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N 
7,ES M7,6/2/2017 10:29,2436.75,VX M7-CF,11.61,Short,VX M7-CF,Short,N,N,N,N 
8,ES M7,6/2/2017 11:29,2436.75,VX M7-CF,11.57,Short,VX M7-CF,Short,Long,N,N,N 
9,ES M7,6/2/2017 12:29,2435.75,VX M7-CF,11.67,Short,VX M7-CF,Short,N,N,N,N 
10,ES M7,6/2/2017 13:29,2437.75,VX M7-CF,11.64,Short,VX M7-CF,Short,N,N,N,N 
11,ES M7,6/2/2017 14:29,2437.75,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N