2017-10-20 137 views
0

我有一個由嵌套的內置原語(列表,字典)和不在項目中的類的實例組成的pickled結構,因此在取出過程中會導致錯誤。我並不關心這些對象,我希望能夠提取存儲在這個嵌套結構中的數值。有沒有辦法從文件中取消選中,並替換因導入問題導致的所有內容,例如None如何在pickle加載過程中將導致導入錯誤的對象替換爲None?

我試圖從Unpickler繼承和覆蓋find_class(self, module, name)返回Dummy如果類不能被發現,但由於某種原因,我一直在後load reduce越來越TypeError: 'NoneType' object is not callable

class Dummy(object): 
    def __init__(self, *argv, **kwargs): 
     pass 

我想是這樣

class RobustJoblibUnpickle(Unpickler): 
    def find_class(self, _module, name): 
     try: 
      super(RobustJoblibUnpickle, self).find_class(_module, name) 
     except ImportError: 
      return Dummy 
+0

你是說,你可以醃一個對象但不能取消它?這裏更廣泛的任務是什麼? – roganjosh

+0

@roganjosh結構(清單的詞典列表)被醃製了一段時間,從那以後代碼庫發生了顯着變化;現在如果我試圖取消它,我面臨導入錯誤,因爲沒有像那裏使用的類,雖然我不需要存儲在那裏的對象,即使只有數值和字符串,我也會很好;所以,是的,廣泛的問題是,我不能undeckle它 –

回答

0

也許你能趕上在try塊中的例外,做你想做的( 設置一些對象 None 用的虛擬類)?

編輯:

看看這個,我不知道它是做正確的方式,但它似乎很好地工作:

import sys 
import pickle 

class Dummy: 
    pass 

class MyUnpickler(pickle._Unpickler): 
    def find_class(self, module, name): # from the pickle module code but with a try 
     # Subclasses may override this. # we are doing it right now... 
     try: 
      if self.proto < 3 and self.fix_imports: 
       if (module, name) in _compat_pickle.NAME_MAPPING: 
        module, name = _compat_pickle.NAME_MAPPING[(module, name)] 
       elif module in _compat_pickle.IMPORT_MAPPING: 
        module = _compat_pickle.IMPORT_MAPPING[module] 
      __import__(module, level=0) 
      if self.proto >= 4: 
       return _getattribute(sys.modules[module], name)[0] 
      else: 
       return getattr(sys.modules[module], name) 
     except AttributeError: 
      return Dummy 

# edit: as per Ben suggestion an even simpler subclass can be used 
# instead of the above 

class MyUnpickler2(pickle._Unpickler): 
    def find_class(self, module, name): 
     try: 
      return super().find_class(module, name) 
     except AttributeError: 
      return Dummy 

class C: 
    pass 

c1 = C() 

with open('data1.dat', 'wb') as f: 
    pickle.dump(c1,f) 

del C# simulate the missing class 

with open('data1.dat', 'rb') as f: 
    unpickler = MyUnpickler(f) # or MyUnpickler2(f) 
    c1 = unpickler.load() 

print(c1) # got a Dummy object because of missing class 
+1

好吧,你需要以某種方式做「內」unickickle例程繼續undeketling替換對象後用'None' –

+0

看到我的答案編輯。這對你有用嗎? – progmatico

+1

是的,我嘗試過這樣的事情,我的問題可能在別的地方;如果我理解正確,那麼可以使用'super()。find_class(s​​elf,module,name)''try ... catch'塊內的內容來完成它。但我的問題似乎在別的地方。感謝您指出這一點,但。 –