2017-10-22 110 views
0

我一直在閱讀Keras文檔建立一個實現MLP反向傳播我自己的MLP網絡。我熟悉sklearn MLPClassifier,但我想學Keras深學習。以下是第一次嘗試。該網絡有3層1輸入(功能= 64),1輸出和1隱藏。總數是(64,64,1)。輸入是125K樣本(64 DIM)和ynumpy矩陣X是1D numpy二進制類(1,-1):不一致的結果sklearn

# Keras imports 
from keras.models import Sequential 
from sklearn.model_selection import train_test_split 
from keras.layers import Dense, Dropout, Activation 
from keras.initializers import RandomNormal, VarianceScaling, RandomUniform 
from keras.optimizers import SGD, Adam, Nadam, RMSprop 

# System imports 
import sys 
import os 
import numpy as np 
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' 


def train_model(X, y, num_streams, num_stages): 

    ''' 
    STEP1: Initialize the Model 
    ''' 

    tr_X, ts_X, tr_y, ts_y = train_test_split(X, y, train_size=.8) 
    model = initialize_model(num_streams, num_stages) 

    ''' 
    STEP2: Train the Model 
    ''' 
    model.compile(loss='binary_crossentropy', 
        optimizer=Adam(lr=1e-3), 
        metrics=['accuracy']) 
    model.fit(tr_X, tr_y, 
       validation_data=(ts_X, ts_y), 
       epochs=3, 
       batch_size=200) 


def initialize_model(num_streams, num_stages): 

    model = Sequential() 
    hidden_units = 2 ** (num_streams + 1) 
    # init = VarianceScaling(scale=5.0, mode='fan_in', distribution='normal') 
    init_bound1 = np.sqrt(3.5/((num_stages + 1) + num_stages)) 
    init_bound2 = np.sqrt(3.5/((num_stages + 1) + hidden_units)) 
    init_bound3 = np.sqrt(3.5/(hidden_units + 1)) 
    # drop_out = np.random.uniform(0, 1, 3) 

    # This is the input layer (that's why you have to state input_dim value) 
    model.add(Dense(num_stages, 
        input_dim=num_stages, 
        activation='relu', 
        kernel_initializer=RandomUniform(minval=-init_bound1, maxval=init_bound1))) 

    model.add(Dense(hidden_units, 
        activation='relu', 
        kernel_initializer=RandomUniform(minval=-init_bound2, maxval=init_bound2))) 

    # model.add(Dropout(drop_out[1])) 

    # This is the output layer 
    model.add(Dense(1, 
        activation='sigmoid', 
        kernel_initializer=RandomUniform(minval=-init_bound3, maxval=init_bound3))) 

    return model 

的問題是,我得到99%的準確度與相同的數據集Xy當使用MLPClassifier sklearn。但是,Keras給出的準確度差,如下所示:

Train on 100000 samples, validate on 25000 samples 
Epoch 1/3 
100000/100000 [==============================] - 1s - loss: -0.5358 - acc: 0.0022 - val_loss: -0.7322 - val_acc: 0.0000e+00 
Epoch 2/3 
100000/100000 [==============================] - 1s - loss: -0.6353 - acc: 0.0000e+00 - val_loss: -0.7385 - val_acc: 0.0000e+00 
Epoch 3/3 
100000/100000 [==============================] - 1s - loss: -0.7720 - acc: 9.0000e-05 - val_loss: -0.9474 - val_acc: 5.2000e-04 

我不明白爲什麼?我在這裏錯過了什麼嗎?任何幫助表示讚賞。

回答

0

我認爲問題是,你使用的是sigmoid輸出層(綁定到[0,1]),但你的類的詳細信息(1,-1 ),您需要更改輸出值或使用tanh

而且keras層可以有不同的默認參數比sklearn,確保你看看那些文檔。

最後一兩件事,爲kernel_initializer嘗試glorot_uniform,這是一個很好的默認。