2017-08-01 100 views
0

您好我有存在這樣 Files present in parent folder讀取文件

BBC文件夾中的每個子文件夾包含BBC文件夾中的文件文本文件 Text files in sub folders

此代碼可以幫助訪問該文件夾內的文件

class MySentences(object): 
def __init__(self, dirname): 
    self.dirname = dirname 

def __iter__(self): 
    for fname in os.listdir(self.dirname): 
     for line in open(os.path.join(self.dirname, fname)): 
      yield line.split() 
sentences = MySentences('C:/Users/JAYASHREE/Documents/NLP/bbc-fulltext/bbc/business') 

但我想訪問每個子文件夾中的文件。當我這樣做時出現以下錯誤

sentences = MySentences('C:/Users/JAYASHREE/Documents/NLP/bbc-fulltext/bbc') 
IOError         Traceback (most recent call last) 
<ipython-input-29-26fb31de4fec> in <module>() 
     1 sentences = MySentences('C:/Users/JAYASHREE/Documents/NLP/bbc-fulltext/bbc') # a memory-friendly iterator 
----> 2 model = gensim.models.Word2Vec(sentences) 

C:\Users\JAYASHREE\Anaconda2\lib\site-packages\gensim-2.3.0-py2.7-win-amd64.egg\gensim\models\word2vec.pyc in __init__(self, sentences, size, alpha, window, min_count, max_vocab_size, sample, seed, workers, min_alpha, sg, hs, negative, cbow_mean, hashfxn, iter, null_word, trim_rule, sorted_vocab, batch_words, compute_loss) 
    501    if isinstance(sentences, GeneratorType): 
    502     raise TypeError("You can't pass a generator as the sentences argument. Try an iterator.") 
--> 503    self.build_vocab(sentences, trim_rule=trim_rule) 
    504    self.train(sentences, total_examples=self.corpus_count, epochs=self.iter, 
    505      start_alpha=self.alpha, end_alpha=self.min_alpha) 

C:\Users\JAYASHREE\Anaconda2\lib\site-packages\gensim-2.3.0-py2.7-win-amd64.egg\gensim\models\word2vec.pyc in build_vocab(self, sentences, keep_raw_vocab, trim_rule, progress_per, update) 
    575 
    576   """ 
--> 577   self.scan_vocab(sentences, progress_per=progress_per, trim_rule=trim_rule) # initial survey 
    578   self.scale_vocab(keep_raw_vocab=keep_raw_vocab, trim_rule=trim_rule, update=update) # trim by min_count & precalculate downsampling 
    579   self.finalize_vocab(update=update) # build tables & arrays 

C:\Users\JAYASHREE\Anaconda2\lib\site-packages\gensim-2.3.0-py2.7-win-amd64.egg\gensim\models\word2vec.pyc in scan_vocab(self, sentences, progress_per, trim_rule) 
    587   vocab = defaultdict(int) 
    588   checked_string_types = 0 
--> 589   for sentence_no, sentence in enumerate(sentences): 
    590    if not checked_string_types: 
    591     if isinstance(sentence, string_types): 

<ipython-input-28-48533b12127a> in __iter__(self) 
     5  def __iter__(self): 
     6   for fname in os.listdir(self.dirname): 
----> 7    for line in open(os.path.join(self.dirname, fname)): 
     8     yield line.split() 

IOError: [Errno 13] Permission denied: 'C:/Users/JAYASHREE/Documents/NLP/bbc-fulltext/bbc\\business' 

請建議我在代碼

+1

你需要'os.walk' –

回答

1

在您的例子改變你似乎直接在文件夾被觸發拒絕IO錯誤,因爲你試圖調用open(一許可)。另外,你可能會得到由os.listdir()返回的系統文件的名字(比如NTUSER.DAT),你不能依賴於能夠打開這些文件。

這裏有一個的遞歸遍歷目錄樹,從我們能夠成功打開任何文件打印行的例子:

import os 

def print_files_in_dir(dirname): 
    try: 
     for fname in os.listdir(dirname): 
      path = os.path.join(dirname, fname) 
      if os.path.isfile(path): 
       try: 
        for line in open(path, 'r'): 
         print(line) 
       except: 
        pass 
      elif os.path.isdir(path): 
       print_files_in_dir(path) # recurse 
    except: 
     pass 

dirname = 'C:/Users/MackayA' 
print_files_in_dir(dirname) 

或者更簡單地說,你可以使用os.walk()之類COLDSPEED建議:

import os 

dirname = 'C:/Users/MackayA' 

for root, dirs, files in os.walk(dirname): 
    for fname in files: 
     path = os.path.join(root, fname) 
     try: 
      for line in open(path, 'r'): 
       print(line) 
     except: 
      pass 

希望這是有用的