2017-05-31 135 views
0

我運行它採用VGG16神經網絡,通過keras包,分類貓狗的圖像,從Kaggle數據庫中的Python程序。爲此,我使用標準終端命令:python program.py > output.txt。我也嘗試了其他變體,python program.py &> output.txttee命令,python program.py |& tee output.txt,但它似乎並沒有工作。對於第一個命令,我的文本文件只包含:Linux終端輸出導出錯誤

Using TensorFlow backend. 
2017-05-31 13:39:34.218034: W tensorflow/core/platform/cpu_feature_guard.cc:45] 
The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are 
available on your machine and could speed up CPU computations. 
2017-05-31 13:39:34.226941: W tensorflow/core/platform/cpu_feature_guard.cc:45] 
The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are 
available on your machine and could speed up CPU computations. 

但代碼有很多print聲明!所述output.txt文件的所期望的內容是(僅第一4-5線的終端輸出的示出):

Using TensorFlow backend. 
Defining all the path! 

All paths defined! 

Getting mean RGB and creating labels! 

當我只鍵入python program.py其上顯示。部分:

2017-05-31 13:39:34.218034: W tensorflow/core/platform/cpu_feature_guard.cc:45] 
The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are 
available on your machine and could speed up CPU computations. 
2017-05-31 13:39:34.226941: W tensorflow/core/platform/cpu_feature_guard.cc:45] 
The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are 
available on your machine and could speed up CPU computations. 

部分來得晚於終端輸出。我在這裏把我的代碼以供參考,但它是204線長:

import keras 
from keras.models import Sequential, Model 
from keras.layers import Flatten, Dense, Dropout, Input, Activation 
from keras.layers.convolutional import Conv2D, MaxPooling2D, ZeroPadding2D 
from keras.layers.merge import Add 
from keras.optimizers import SGD, Adam 
import cv2, numpy as np 
import glob 
import csv 

#################### 
## VGG16 Function ## 
#################### 

def VGG_16(weights_path=None, classes=2): 

    ###################################### 
    ## Input: 3x224x224 sized RGB Input ## 
    ###################################### 

    inputs = Input(shape=(3,224,224)) 

    layer = 0 
    ############# 
    ## Block 1 ## 
    ############# 
    x = Conv2D(64, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block1_conv1')(inputs) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(64, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block1_conv2')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x) 

    ############# 
    ## Block 2 ## 
    ############# 
    x = Conv2D(128, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block2_conv1')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(128, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block2_conv2')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x) 

    ############# 
    ## Block 3 ## 
    ############# 
    x = Conv2D(256, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block3_conv1')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(256, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block3_conv2')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(256, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block3_conv3')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x) 

    ############# 
    ## Block 4 ## 
    ############# 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block4_conv1')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block4_conv2')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block4_conv3')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x) 

    ############# 
    ## Block 5 ## 
    ############# 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block5_conv1')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block5_conv2')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    x = Conv2D(512, (3, 3), data_format='channels_first', activation='relu', padding='same', name='block5_conv3')(x) 
    layer += 1 
    print ('Output shape for Layer ' +str(layer)+ ', is, ' +str(x.get_shape())) 
    out = MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x) 

    ############### 
    ## Top layer ## 
    ############### 

    out = Flatten(name='flatten')(out) 
    out = Dense(4096, activation='relu', name='fc1')(out) 
    out = Dropout(0.5)(out) 
    out = Dense(4096, activation='relu', name='fc2')(out) 
    out = Dropout(0.5)(out) 
    out = Dense(classes, activation='softmax', name='predictions')(out) 

    if weights_path: 
     model.load_weights(weights_path) 

    model = Model(inputs, out, name='vgg-16') 

    return model 

################### 
## Main Function ## 
################### 

if __name__ == "__main__": 

    ################################################ 
    ## Get all the training and the testing paths ## 
    ################################################ 

    print('Defining all the path!\n') 
    cat_path = "./train/cat.*.jpg" 
    dog_path = "./train/dog.*.jpg" 
    train_path = "./train/*.jpg" 
    test_path = "./test1/*.jpg" 
    Mean_RGB = [] 
    x_train = [] 
    y_train = [] 
    x_test = [] 
    print('All paths defined!\n') 

    ######################################################################## 
    ## Get training and testng data sizes, to find the average RGB values ## 
    ######################################################################## 

    print('Getting mean RGB and creating labels!\n') 
    for file in glob.glob(cat_path): # To get the sizes of all the cat images 
     im = cv2.resize(cv2.imread(file), (224, 224)).astype(np.float32) 
     im = np.mean(im, axis=(0,1)) 
     Mean_RGB.append(tuple(im)) 
     y_train.append(0) 
    for file in glob.glob(dog_path): # To get the sizes of all the dog images 
     im = cv2.resize(cv2.imread(file), (224, 224)).astype(np.float32) 
     im = np.mean(im, axis=(0,1)) 
     Mean_RGB.append(tuple(im)) 
     y_train.append(1) 
    y_train = np.array(y_train) 
    Mean_RGB = tuple(np.mean(Mean_RGB, axis=0)) 
    print('Got mean RGB and created labels!\n') 

    ######################################################################### 
    ## Load the training and testing images, after subtracting average RGB ## 
    ######################################################################### 

    print('Loading images as numpy arrays!\n') 
    for file in glob.glob(train_path): 
     im = cv2.resize(cv2.imread(file), (224, 224)).astype(np.float32) 
     im_r = im-Mean_RGB 
     im_r = im_r.transpose((2,0,1)) 
     #im_r = np.expand_dims(im_r, axis=0) 
     x_train.append(im_r) 
    y_train = y_train.reshape((-1,1)) 
    y_train = keras.utils.to_categorical(y_train, num_classes=2) 
    x_train = np.array(x_train) 
    for file in glob.glob(test_path): 
     im = cv2.resize(cv2.imread(file), (224, 224)).astype(np.float32) 
     im_r = im-Mean_RGB 
     im_r = im_r.transpose((2,0,1)) 
     #im_r = np.expand_dims(im_r, axis=0) 
     x_test.append(im_r) 
    x_test = np.array(x_test) 
    print('All images loaded!\n') 

    ############################## 
    ## Train and test the model ## 
    ############################## 

    print('Creating Neural Net!\n') 
    model = VGG_16() 
    print('\nNeural Net created!\n') 
    adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0) 
    model.compile(optimizer=adam, loss='binary_crossentropy', metrics=['accuracy']) 

    print('Training Neural Net!\n') 
    ### Generating validation data split in training sample 
    model.fit(x_train, y_train, batch_size=500, epochs=25, validation_split=0.2, shuffle=True) 
    print('Neural Net trained!\n') 
    print('Evaluating model on the training images!\n') 
    score = model.evaluate(x_train, y_train, batch_size=500, verbose=1) 
    print('Model score on training data: ' +str(score)+ '\n') 
    print('Predicting class of test images!\n') 
    pred = model.predict(x_test, batch_size=1, verbose=1) 
    prediction = np.argmax(pred, axis = 1) 
    print('Predictions done!\n') 
    result = [] 
    print('Creating output CSV file!\n') 
    result.append(['id', 'label']) 
    for i in range(0,len(prediction)): 
     result.append([i+1,prediction[i]]) 
    with open("cat-dog-output.csv","wb") as f: 
     writer = csv.writer(f) 
     writer.writerows(result) 
    print('Created output CSV file!\n') 

    print('Saving model parameters!\n') 
    model.save('vgg16-sim-conn.h5') 
    model.save_weights('vgg16-sim-conn-weights.h5') 
    print('Model saved!\n') 

我不知道究竟怎麼回事上,並在這個問題上的任何幫助將十分讚賞!

+0

你可以創建一個最小的例子來顯示你的系統上我們實際上可以重現的行爲嗎?就目前而言,這個問題很可能是無法回答的,因爲除了您之外,沒有人可以訪問該系統,並且您沒有顯示任何代碼。 –

+0

您是否嘗試過使用2>來輸出標準錯誤? –

+0

@MadPhysicist代碼長度爲203行。我將它添加到我的帖子中,但這使得帖子非常長。 – Prabaha

回答

0

經過對python命令和幫助python -h的一些修補之後,我發現有一個用於非緩衝輸出的選項-u。我試了一下,python -u program.py > tee output.txt,它工作完美。
我已經發布了my question in Ask Ubuntu以及Steven D.也提出了相同的解決方案。他的回答也將我重定向到similar question in StackOverflow