1
我試圖總結一個數據分析項目,它運行許多ipython/jupyter筆記本,每個筆記本都相當長。有助於這個過程的事情之一是,如果我至少知道整個「投入」醬菜進入和「輸出」泡菜走出去。提取進出ipython/jupyter筆記本的泡菜的方法
什麼是最乾淨/最快/最有效的方式來做到這一點?
我試圖總結一個數據分析項目,它運行許多ipython/jupyter筆記本,每個筆記本都相當長。有助於這個過程的事情之一是,如果我至少知道整個「投入」醬菜進入和「輸出」泡菜走出去。提取進出ipython/jupyter筆記本的泡菜的方法
什麼是最乾淨/最快/最有效的方式來做到這一點?
我不知道這是否是做的最好的方式,但它至少有一種方式......
def summerize_pickles(notebook_path):
from IPython.nbformat import current as nbformat
import re
with open(notebook_path) as fh:
nb = nbformat.reads_json(fh.read())
list_of_input_pickles = []
list_of_output_pickles = []
for cell in nb["worksheets"][0]["cells"]:
# This confirms there is at least one pickle in it.
if cell["cell_type"] != "code" or cell["input"].find("pickle") == -1: # Skipping over those cells which aren't code or those cells with code but which don't reference "pickle
continue
# In case there are multiple lines, it iterates line by line.
for line in cell["input"].splitlines():
if line.find("pickle") == -1: # Skips over lines w/ no mention of "pickle" to potentially reduce the number of times it's searched.
continue
############################ ############################ ############################ ############################
code_type = str()
if line.find("pickle.dump") != -1 or line.find(".to_pickle")!= -1:
code_type = "output"
elif line.find("pickle.load") != -1 or line.find(".read_pickle")!= -1:
code_type = "input"
else:
continue # This tells the code to skip over lines like "import cpickle as pickle"
############################ ############################ ############################ ############################
filename = re.findall(r'"(.*?)"', line) # This gets all the content between the quotes. See: http://stackoverflow.com/questions/171480/regex-grabbing-values-between-quotation-marks
############################ ############################ ############################ ############################
if code_type == "input":
list_of_input_pickles.append(filename[0])
elif code_type == "output":
list_of_output_pickles.append(filename[0])
pickles_dict = {"input_pickles":list_of_input_pickles,
"output_pickles":list_of_output_pickles }
return pickles_dict
這並不與鹹菜幫助,但它確實在看到結構幫助筆記本電腦:查看Calico工具和其他反應在這裏:http://stackoverflow.com/questions/24895714/is-it-possible-to-create-grouping-of-input-cells-in-ipython-notebook – Afflatus