pandas在傳遞到kdb +時（使用qPython API）丟失索引

Question

我試圖將時間序列數據從Python傳遞到q/kdb+。

一個解決方案是qPython module，提供從q表/字典到Pandas的無縫轉換。

的問題是試圖從熊貓傳遞到q，時間指數DataFrame（列Date）時並不完全使它的q側。可重複碼：

import pandas.io.data as web 
import datetime 
import numpy 
import qpython.qconnection as qconnection # requires installation of qPython module from https://github.com/exxeleron/qPython 

start = datetime.datetime(2010, 1, 1) 
end = datetime.datetime(2015, 2, 6) 
f=web.DataReader("F", 'yahoo', start, end) # download Ford stock data (ticker "F") from Yahoo Finance web service 
f.ix[:5] # explore first 5 rows of the DataFrame 
# Out: 
#    Open High Low Close Volume Adj Close 
# Date 
# 2010-01-04 10.17 10.28 10.05 10.28 60855800  9.43 
# 2010-01-05 10.45 11.24 10.40 10.96 215620200  10.05 
# 2010-01-06 11.21 11.46 11.13 11.37 200070600  10.43 
# 2010-01-07 11.46 11.69 11.32 11.66 130201700  10.69 
# 2010-01-08 11.67 11.74 11.46 11.69 130463000  10.72 

q = qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) # define connection interface parameters. Assumes we have previously started q server on port 5000 with `q.exe -p 5000` command 
q.open() # open connection 
q('set', numpy.string_('yahoo'), f) # pass DataFrame to q table named `yahoo` 
q('5#yahoo') # display top 5 rows from newly created table on q server 
# Out: 
# Open High Low Close Volume Adj Close 
# 0 10.17 10.28 10.05 10.28 60855800  9.43 
# 1 10.45 11.24 10.40 10.96 215620200  10.05 
# 2 11.21 11.46 11.13 11.37 200070600  10.43 
# 3 11.46 11.69 11.32 11.66 130201700  10.69 
# 4 11.67 11.74 11.46 11.69 130463000  10.72 

正如你所看到的，問表不具備Date列存在於f數據幀的指數。

如何有效地（對於大數據）將日期時間索引傳遞給q？

Answer 1

在序列化DataFrame對象時，qPython檢查是否存在meta屬性。如果該屬性不存在，則DataFrame被序列化爲q表，並且索引列在該過程中被跳過。如果要保留索引列，則必須設置meta屬性並提供類型提示以強制執行q鍵表的表示。

請看看修改後的樣本：

import pandas.io.data as web 
import datetime 
import numpy 
import qpython.qconnection as qconnection # requires installation of qPython module from https://github.com/exxeleron/qPython 

from qpython import MetaData 
from qpython.qtype import QKEYED_TABLE 


start = datetime.datetime(2010, 1, 1) 
end = datetime.datetime(2015, 2, 6) 
f=web.DataReader("F", 'yahoo', start, end) # download Ford stock data (ticker "F") from Yahoo Finance web service 
f.ix[:5] # explore first 5 rows of the DataFrame 
# Out: 
#    Open High Low Close Volume Adj Close 
# Date 
# 2010-01-04 10.17 10.28 10.05 10.28 60855800  9.43 
# 2010-01-05 10.45 11.24 10.40 10.96 215620200  10.05 
# 2010-01-06 11.21 11.46 11.13 11.37 200070600  10.43 
# 2010-01-07 11.46 11.69 11.32 11.66 130201700  10.69 
# 2010-01-08 11.67 11.74 11.46 11.69 130463000  10.72 

q = qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) # define connection interface parameters. Assumes we have previously started q server on port 5000 with `q.exe -p 5000` command 
q.open() # open connection 
f.meta = MetaData(**{'qtype': QKEYED_TABLE}) # enforce to serialize DataFrame as keyed table 
q('set', numpy.string_('yahoo'), f) # pass DataFrame to q table named `yahoo` 
q('5#yahoo') # display top 5 rows from newly created table on q server 
# Out: 
#    Open High Low Close  Volume Adj Close 
# Date               
# 2010-01-04 10.17 10.28 10.05 10.28 60855800  9.43 
# 2010-01-05 10.45 11.24 10.40 10.96 215620200  10.05 
# 2010-01-06 11.21 11.46 11.13 11.37 200070600  10.43 
# 2010-01-07 11.46 11.69 11.32 11.66 130201700  10.69 
# 2010-01-08 11.67 11.74 11.46 11.69 130463000  10.72 

Answer 2

您是否考慮嘗試其他API之一？ http://www.timestored.com/kdb-guides/python-api 我也會在他們的github頁面上報告問題。