蟒蛇合併多個xml到單個CSV

Question

這是我試圖放在一起讓我的生活更輕鬆的腳本片段。我有一堆來自不同API源的XML文件。他們有不同的項目和不同數量的領域。他們所有的共同點是像「clientid」這樣的共同領域。

我想要做的是結束一個CSV，它包含所有XML與其對應數據的組合標題。所以我需要能夠確保所有從12345「客戶端ID」的信息添加到該行對同一名稱中的「爲itemid」的客戶端id的端部拉手

項目數據：

腳本的

<item> 
<id>99899</id> 
<client-id>12345</client-id> 

部分：

def parseXML(): 
    ### Parse XML and convert to CSV ### 
    #Get XML Source # 
    tree = ET.fromstring(getdata) 
    # open a file for writing 
    xmlTest01 = open('xmlTest01.csv', 'w') 
    # create the csv writer object 
    csvwriter = csv.writer(xmlTest01) 
    item_head = [] 

    count = 0 
    for member in tree.findall('item'): 
        item = [] 
        if count == 0: 
            id = member.find('id').tag 
            item_head.append(id) 
            clientid = member.find('client-id').tag 
            item_head.append(clientid) 

        id = member.find('id').text 
        item.append(id) 
        clientid = member.find('client-id').text 
        item.append(clientid) 
        csvwriter.writerow(item) 
    xmlTest01.close() 

下一組數據有這它：

<client> 
<id>12345</id> 
<name>Clients name</name> 
<current type="boolean">true</current> 
<status>good</status> 

所以我要到c將前一組數據中的行與相同的clientid相比較，然後將名稱，當前和狀態添加到該行的末尾。

有關最佳方式的任何想法？我有大約5-7這些類型的文件合併。我應該在將它們轉換爲CSV之前先將這些文件合併在一起？這可能是好的，如果他們都有類似的內容，但他們沒有。

所需的輸出，其結合了XML文件的值：

id,clientid,name,current,status 
99899,12345,Clients name,true,good 

Answer 1

考慮在這三個文件迭代和有條件檢查客戶端ID。解析XML值，你寫csv文件列表：

import csv 
import xml.etree.ElementTree as ET 

def parseXML(): 
    projecttree = ET.parse('projects.xml') 
    clienttree = ET.parse('clients.xml') 
    teamtasktree = ET.parse('teammembers.xml') 

    projectroot = projecttree.getroot() 
    clientroot = clienttree.getroot() 
    teamtaskroot = teamtasktree.getroot() 

    data = [] 
    for i in projectroot.iter('project'): 
     for j in clientroot.iter('client'): 
      clientid = i.find('client-id').text 
      if clientid == j.find('id').text: 
       data.append(i.find('id').text) 
       data.append(j.find('id').text) 
       data.append(j.find('name').text) 
       data.append(j.find('active').text) 
       data.append(i.find('name').text) 
       data.append(i.find('active').text) 
       data.append(i.find('billable').text) 
       data.append(i.find('bill-by').text) 
       data.append(i.find('hourly-rate').text) 
       data.append(i.find('budget').text) 
       data.append(i.find('over-budget-notification-percentage').text) 
       data.append(i.find('created-at').text) 
       data.append(i.find('updated-at').text) 
       data.append(i.find('starts-on').text) 
       data.append(i.find('ends-on').text) 
       data.append(i.find('estimate').text) 
       data.append(i.find('estimate-by').text) 
       data.append(i.find('notes').text) 
       data.append(i.find('cost-budget').text) 

     cnt = 1  
     for tm in teamtaskroot.iter('team_members'): 
      for item in tm.iter('item'):     
       if item.find('cid').text == clientid and cnt <= 3: 
        data.append(item.find('full_name').text) 
        data.append(item.find('cost_rate').text) 
       cnt += 1 

     cnt = 1 
     for tk in teamtaskroot.iter('tasks'): 
      for item in tk.iter('item'):     
       if item.find('cid').text == clientid and cnt <= 2: 
        data.append(item.find('task_id').text) 
        data.append(item.find('total_hours').text) 
       cnt += 1 

    with open('Output.csv', 'w') as f: 
     csvwriter = csv.writer(f, lineterminator = '\n') 
     csvwriter.writerow(['Pid', 'Clientid', 'ClientName', 'ClientActive', 'ProjectName', 'ProjectActive', 
          'Billable', 'BillBy', 'HourlyRate', 'Budget', 'OverbudgetNotificationPercentage', 
          'CreatedAt', 'UpdatedAt', 'StartsOn', 'EndsOn', 'Estimate', 'EstimateBy', 
          'Notes', 'CostBudget', 'TeammemberName1', 'CostRate1', 'TeammemberName2', 'CostRate2', 
          'TeammemberName3', 'CostRate3', 'TaskId1', 'TotalHours1', 'TaskId2', 'TotalHours2']) 
     csvwriter.writerow(data) 

if __name__ == "__main__": 
    parseXML() 

輸出

Pid,Clientid,ClientName,ClientActive,ProjectName,ProjectActive,Billable, 
BillBy,HourlyRate,Budget,OverbudgetNotificationPercentage,CreatedAt, 
UpdatedAt,StartsOn,EndsOn,Estimate,EstimateBy,Notes,CostBudget,TeammemberName 
1,CostRate1,TeammemberName2,CostRate2,TeammemberName3,CostRate3, 
TaskId1,TotalHours1,TaskId2,TotalHours2 
11493770,4708336,AFB,true,Services - Consulting - AH,true,true,Project, 
421.28,16.0,80.0,2016-08-16T03:22:51Z, 
2016-08-16T03:22:51Z,,,16.0,project,Random 
notes,,BobR,76.0,BobR,76.0,BobR,76.0,6357137,0.0,6357138,0.0 

Answer 2

此外，考慮XSLT，特殊用途的轉換語言，可以直接將XML轉換爲CSV甚至從解析其他XML文件使用其document()函數。 Python的lxml模塊可以處理XSLT 1.0腳本。確保所有三個xmls都駐留在同一個目錄中。

XSLT腳本（保存爲文件名爲.xsl --a特別.XML file--要在Python下面的稱呼）

<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> 
<xsl:output version="1.0" encoding="UTF-8" method="text" indent="yes" omit-xml-declaration="yes"/> 
<xsl:strip-space elements="*"/> 

    <xsl:template match="/projects"> 
    <xsl:copy> 
     <xsl:text>Pid,Clientid,ClientName,ClientActive,ProjectName,ProjectActive,Billable,BillBy,HourlyRate,</xsl:text> 
     <xsl:text>Budget,OverbudgetNotificationPercentage,CreatedAt,UpdatedAt,StartsOn,EndsOn,Estimate,EstimateBy,</xsl:text> 
     <xsl:text>Notes,CostBudget,TeammemberName1,CostRate1,TeammemberName2,CostRate2,TeammemberName3,CostRate3,</xsl:text> 
     <xsl:text>TaskId1,TotalHours1,TaskId2,TotalHours2&#xa;</xsl:text>  
     <xsl:apply-templates select="project"/>  
    </xsl:copy> 
    </xsl:template> 

    <xsl:template match="project">  
    <xsl:variable name="clientid" select="client-id"/> 
    <xsl:value-of select="concat(id, ',')"/> 
    <xsl:variable name="delimiter"><xsl:text>&quot;,&quot;</xsl:text></xsl:variable> 

    <xsl:for-each select="document('clients.xml')/clients/client[id=$clientid]/* 
            [local-name()='id' or local-name()='name' or local-name()='active']"> 
     <xsl:value-of select="." /> 
     <xsl:if test="position() != last()"> 
     <xsl:text>,</xsl:text> 
     </xsl:if>  
    </xsl:for-each> 

    <xsl:value-of select="concat(',',name,',',active,',',billable,',',bill-by,',',hourly-rate,',',budget,',', 
           over-budget-notification-percentage,',',created-at,',',updated-at,',',starts-on,',',ends-on,',', 
           estimate,',',estimate-by,',',notes,',',cost-budget,',')"/>   

    <xsl:for-each select="document('teammembers.xml')/root/team_members/item[cid=$clientid]/* 
            [local-name()='full_name' or local-name()='cost_rate']"> 
     <xsl:if test="position() &lt; 5"> 
     <xsl:value-of select="." />  
     <xsl:text>,</xsl:text> 
     </xsl:if> 
    </xsl:for-each> 

    <xsl:for-each select="document('ClientItems_teammembers.xml')/root/tasks/item[cid=$clientid]/* 
            [local-name()='task_id' or local-name()='total_hours']"> 
     <xsl:if test="position() &lt; 5"> 
     <xsl:value-of select="." /> 
     <xsl:if test="position() != last()"> 
      <xsl:text>,</xsl:text> 
     </xsl:if> 
     </xsl:if> 
    </xsl:for-each> 

    <xsl:text>&#xa;</xsl:text> 
    </xsl:template> 

</xsl:transform> 

的Python腳本（轉化projects.xml和讀取在XSLT另外兩個）

import lxml.etree as ET 

def transformXML(): 
    dom = ET.parse('projects.xml') 
    xslt = ET.parse('XSLTscript.xsl') 

    transform = ET.XSLT(xslt) 
    newdom = transform(dom) 

    with open('Output.csv'),'w') as f: 
     f.write(str(newdom)) 

if __name__ == "__main__": 
    transformXML() 

輸出

Pid,Clientid,ClientName,ClientActive,ProjectName,ProjectActive,Billable, 
BillBy,HourlyRate,Budget,OverbudgetNotificationPercentage,CreatedAt, 
UpdatedAt,StartsOn,EndsOn,Estimate,EstimateBy,Notes,CostBudget,TeammemberName 
1,CostRate1,TeammemberName2,CostRate2,TeammemberName3,CostRate3, 
TaskId1,TotalHours1,TaskId2,TotalHours2 
11493770,4708336,AFB,true,Services - Consulting - AH,true,true,Project, 
421.28,16.0,80.0,2016-08-16T03:22:51Z, 
2016-08-16T03:22:51Z,,,16.0,project,Random 
notes,,BobR,76.0,BobR,76.0,BobR,76.0,6357137,0.0,6357138,0.0,