未驗證數據的最佳方式是什麼？

Question

我有一個非常大的數據集（stackoverflow的數據轉儲之一），這是完全原始和消毒形式。

For example: </p> 

是否有已經建立的方式將上述和類似的內容轉換回其原始形式以提高可讀性和可用性？一個偶然的python腳本或函數調用？

Answer 1

這裏是一個解決方案，我不得不用得到的一切工作正常 - 注意，HTML解析器沒有盡全力，我想用我的數據集

在/ usr/bin中/ python3

import html.parser 
    import string 
    import sys 

    # Amount of lines to put into a buffer before writing 
    BUFFER_SIZE_LINES = 1024 
    html_parser = html.parser.HTMLParser() 

    # Few HTML reserved chars that are not being cleaned up by HTMLParser 
    dict = {} 
    dict[ '"' ] = '"' 
    dict[ ''' ] = "'" 
    dict[ '&' ] = '&' 
    dict[ '&lt;' ] = '<' 
    dict[ '&gt;' ] = '>' 

    # Process the file 
    def ProcessLargeTextFile(fileIn, fileOut): 
     r = open(fileIn, "r") 
     w = open(fileOut, "w") 
     buff = "" 
     buffLines = 0 
     for lineIn in r: 

      lineOut = html_parser.unescape(lineIn) 
      for key, value in dict.items(): 
       lineOut = lineOut.replace(key,value) 

      buffLines += 1 

      if buffLines >= BUFFER_SIZE_LINES: 
       w.write(buff) 
       buffLines = 1 
       buff = "" 

      buff += lineOut + "\n" 

     w.write(buff) 
     r.close() 
     w.close() 


    # Now run 
    ProcessLargeTextFile(sys.argv[1],sys.argv[2]) 

！