長期數據採集

Question

我想運行一個Python腳本，是爲長期實驗室數據採集（〜1個月）。下面的腳本應該從SQL數據庫獲取數據，並從DAQ設備獲取數據並將其保存到文本文件中。然後實時繪製1小時的數據。實時繪圖由於記憶錯誤而被註釋掉了，但這不是我關心的問題。

運行了3-4天之間的代碼給我下面的錯誤：

Trackback (most recent call last): 
file *** line 105 in <module> 
    deltay=float(float(tupline[3]/10-float(tupline2[3])/10) 
TypeError: 'NoneType' object has to attribute '__getitem__' 

我如何能得到這個腳本對於較長時間運行有什麼想法？

import ctypes 
from UniversalLibrary import * 
import time 
import datetime 
from numpy import * 
from scipy import * 
import MySQLdb as mdb 
import matplotlib 
matplotlib.use('Agg') 
import matplotlib.pyplot as plt 
import random 



boardNum=0 
portNum=10 
inputVoltage = 3 



BoardNum = 0 
Gain = BIP5VOLTS 
Chan = 0 



spTimes =14*24*60*60  ##here you set the time 
timeDuration=spTimes 
padsToVisualize =[0,1,2,3,4] ##pads that will be measured and visualized 

#plt.ion() ##this command is necassary in order to have new points on the plot without erasing the old ones 

curTime=time.clock() 
movingWinDur=60 ##seconds 
print "curtime=",curTime 

str_one_line=[] 
straverage=[] 
while (curTime < timeDuration): 
    del str_one_line[:] 
    padCounter=0; 

    d=datetime.datetime.now() 
    str_one_line=[(d.strftime("%Y-%m-%d %H:%M:%S")),','] ##formatting the date -- this format is chosen based on the requirements for connecting to SQL database 

    while (padCounter <len(padsToVisualize)): 

     #fig1=plt.figure(padsToVisualize[padCounter]) ## making figures for plotting different pads 


     print "pad No.: ", str(padsToVisualize[padCounter]) 
     #l=cbAOut(BoardNum,Chan,Gain,3019) ## sets the voltage to 3.0 volts (the number 3019 is based on 12-bit resolution --> DAQ resolution) 
     #n=cbDConfigPort(BoardNum,portNum,1) ##the variable n serves aas a command line where the command is sent to the DAQ 
     #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter])) #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals 
     curTime=time.clock() 
     d=datetime.datetime.now() 
     ######################################################################### 

     ########## this part will use the datetime to fetch the temperature from the SQL data base 
     ########## since the time stamps may not exactly match, the code uses a method of interpolation 
     freq = 5 
     time.sleep(1.8*freq) ## 1.8 is an arbitrary number ; to make sure there is data on sql database 
     searchfor=d ## the following lines find the temperature at time=searchfor 
     print searchfor 
     minussearchfor = searchfor-datetime.timedelta(0,freq/1.2) 
     STRminussearchfor = minussearchfor.strftime("%Y-%m-%d %H:%M:%S") 
     print "STRminussearchfor=", STRminussearchfor 
     print "minussearchfor= ", minussearchfor 
     plussearchfor =searchfor+datetime.timedelta(0,freq/1.2) 
     print "plussearchfor= ", plussearchfor 
     STRplussearchfor = plussearchfor.strftime("%Y-%m-%d %H:%M:%S") 
     print "STRplussearchfor=", STRplussearchfor 
     ##Database connection 
     db = mdb.connect("localhost", "root", "qwerty@12345", "bondtest") 
     cur = db.cursor() 
     cur.execute("SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime",{'date1':STRminussearchfor,'date2':STRplussearchfor}) 
##  con=pymssql.connect(host='LAB10-PC\SQLEXPRESS2008R2',user='sa',password='qwerty@12345') 
##  cur = con.cursor() 
##  cur.execute('SELECT * FROM OVEN11SQL.dbo.OvenLog1 WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRminussearchfor,'date2':STRplussearchfor}) 
     tupline1 = cur.fetchone() 
##  print 'between1= ',tupline1[1] 
     delta = tupline1[1]-searchfor 
##  print "delta = " ,delta 
     if (int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S"))>0): 
      delta = int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S")) 
     else: 
      delta = int(tupline1[1].strftime("%S"))-int(searchfor.strftime("%S")) 
##  print 'delta= ',delta 
     time1=tupline1[1]-datetime.timedelta(0,freq/1.2) 
     STRtime1=time1.strftime("%Y-%m-%d %H:%M:%S") 
     time2=tupline1[1]-datetime.timedelta(0,3*freq/1.2) 
     STRtime2=time2.strftime("%Y-%m-%d %H:%M:%S") 
##  time.sleep(2*freq) ##the program needs to wait for 3*frequency/2 so that the next point is available in SQL data base for interpolation 
     cur.execute('SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRtime2,'date2':STRtime1}) 
     tupline2 = cur.fetchone() ##next point is fetched in order to find the slope of the line for temperature change 
##  print 'between2= ', tupline2[1] 
     deltay=float(float(tupline1[3])/10-float(tupline2[3])/10) 
     deltax = int(tupline1[1].strftime("%S"))-int(tupline2[1].strftime("%S")) 
     deltax = freq 
##  print "deltay= ", deltay 
##  print "deltax= ", deltax 
     slope = deltay/deltax 
##  print 'slope= ', slope 

     ##in the following if statements depending on whether the temperature is increasing or decreasing the temperature for the desired point will be calculated 
     if (tupline2[3]<tupline1[3]): 
      tempsearchfor = float(tupline1[3])/10+delta*slope 
##   print '+delta*slope= ',delta*slope 
     elif (tupline2[3]>tupline1[3]): 
      tempsearchfor = float(tupline1[3])/10-delta*slope 
##   print '-delta*slope= ',delta*slope 
     else: 
      tempsearchfor = float(tupline1[3])/10 
##  print 'tempserachfor= ',tempsearchfor 

     ######################################################################### 

     strng = [str(int(padsToVisualize[padCounter])),',',(d.strftime("%Y-%m-%d %H:%M:%S")),',',str(round(curTime,4)),',',str(inputVoltage),','] 
     str_one_line.extend(strng)   ##str_one_line is the line that contains the values that will be written to the text file , the order is specified in the variables lists excel file 
     xyzCounter=Chan 
     EngUnits= array(range(50*1),dtype=float).reshape(50,1) ## constructing the array that will hold the 50 values for each set of signals and is later used for finding the average 
     average = array(range(3*1),dtype=float).reshape(3,1)  ## holds the average of the t50 points for x,y,z 
##  straverage=array(range(3*1),dtype=str).reshape(3,1) ##this array will hold the strings for x,y,z signal to be written into txtfile 
     del straverage[:] 
     while(xyzCounter<3): ##the way the oven is set up it has x,y,z outputs from channel 0,1,2 accordingly 
      #n=cbDConfigPort(BoardNum,portNum,1) 
      #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter]))  #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals 
      a=0 
      time1=time.clock() 
      while (a<50): ## this while loop takes 50 measurements and writes the average of those points in to average array 

       #DataValue=cbAIn(BoardNum, xyzCounter, Gain) 
       #EngUnits[a,0]=float((cbToEngUnits(BoardNum,Gain,DataValue))/3/100*1000) 
       EngUnits[a,0] = random.uniform(0,0.5) 
       average[xyzCounter,0]=float(sum(EngUnits))/len(EngUnits) 
       a+=1 
      time2=time.clock() 
      timePerSample=(time2-time1)/50 
      print "time per sample= ",timePerSample 
      print "samples per second =", (1/timePerSample) ##measuring the frequency of the emasurements 
      tempstr=str(round((average[xyzCounter,0]),3)) ##in order to remove the two brackets temp refers to temporary 
      #tempstr=tempstr[1:-1] 
      straverage.append(tempstr) 
      xyzCounter+=1 


     #print average 
     temperaturearray=array(range(1*1),dtype=str).reshape(1,1) 
     temperaturearray=[str(tempsearchfor)] 
     three_sens_signals=array(range(1*5),dtype=str).reshape(1,5) 
     three_sens_signals=[str((straverage[0])),',',str((straverage[1])),',',str((straverage[2])),','] 
     str_one_line.extend(three_sens_signals) 
     str_one_line.extend(temperaturearray) 
     str_one_line.extend(',') 

     padCounter+=1 
     filename='log_simulation.txt' 
     f = open(filename,"a") 
     ## writing to file section 
    print "padcounter=",padCounter," str_one_line=", str_one_line 
    for item in str_one_line: 
     f.write("%s" % item) 
    f.write("\n") 
    f.close() 

    curTime=time.clock() 

Answer 1

總之，有很多需要加以改進/這段代碼返工的事情（編輯所有進口*語句和使用的命名空間應該像你開始）。話雖如此（代碼顯然大多是按原樣工作），但問題的發生是因爲無論出於何種原因，存儲在tupline或tupline2中的值都是None。 （無論何種原因的數據或表損壞），這會導致它返回「無」，而且在你的代碼，這些變量是使用SQL語句

tupline = cur.fetchone() 
tupline2 = cur.fetchone() 

其中一個電話顯然是不順利運行分配。也許有人正在刪除或替換表格？您可以添加一些語義來檢查這種情況，報告並嘗試繼續。一些東西沿線

if tupline == None or tupline2 == None: 
    print "Why does tupline = None?" 
    continue 

你應該弄清sql .fetchone（）方法何時返回無。真的，你應該做一些更好的日誌記錄，但是這可能會讓你進入下一個調試階段。

Answer 2

正如Paul Seeb所述，錯誤信息發生是因爲tupline或tupline2是None，當代碼到達deltay=...行時。您的代碼（成功）在分配後訪問tupline元素，然後到達deltay行。所以問題必須與tupline2。

我的猜測是數據庫不包含tupline2查詢的匹配記錄;這肯定是要檢查的東西。我還注意到，您在tupline2查詢之前發佈了延遲註釋，表明此查詢不能可靠地返回記錄。我不知道你的程序邏輯，但你也許能夠只用tupline2查詢後，像這樣的線路解決該問題：

if tupline2 is None: 
    continue` 

，或者直到它成功

while True: 
    cur.execute(
     """SELECT * FROM bondtest_data 
      WHERE DateTime BETWEEN %(date1)s AND %(date2)s 
      ORDER BY DateTime""", 
     {'date1':STRtime2,'date2':STRtime1} 
    ) 
    tupline2 = cur.fetchone() 
    if tupline2 is None: 
     print "Failed to obtain a record for tupline2; trying again..." 
     time.sleep(<some duration>) 
    else: 
     break 

你可能會重試 tupline2查詢

我還注意到，您的代碼每次通過循環時都會創建一個新的數據庫連接（db），並且永遠不會明確地關閉它。我建議在主循環開始時移動db = mdb.connect(...)命令，或者在最後的fetchone()命令之後的某處添加db.close()。你的代碼有可能達到某種連接限制，但這可能會引發一個不同的例外。當db被重新分配並且舊連接超出範圍時，更可能您從某種自動關閉連接中受益。