如何將Keras模型插入scikit-learn pipeline？

Question

我正在使用Scikit-Learn自定義管道（sklearn.pipeline.Pipeline）與RandomizedSearchCV一起進行超參數優化。這很好。

現在我想插入Keras模型作爲管道的第一步。模型的參數應該被優化。計算出來的（擬合的）Keras模型應該稍後在其他步驟中用於流水線中，所以我認爲我必須將模型存儲爲全局變量，以便其他流水線步驟可以使用它。這是正確的嗎？我知道Keras爲Scikit-Learn API提供了一些包裝，但問題是這些包裝已經做了分類/迴歸，但我只想計算Keras模型而沒有其他。

這怎麼辦？

例如，我有它返回模型的方法：

def create_model(file_path, argument2,...): 
    ... 
    return model 

的方法需要某些固定參數等的文件路徑等，但不需要的X和Y（或可以忽略）。應該優化模型的參數（層數等）。

Answer 1

這是RBM例子sklearn文檔（http://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#sphx-glr-auto-examples-neural-networks-plot-rbm-logistic-classification-py）

的修改，但在keras與tensorflow後端實現的神經網絡

# -*- coding: utf-8 -*- 
    """ 
    Created on Mon Nov 27 17:11:21 2017 

    @author: ZED 
    """ 

    from __future__ import print_function 

    print(__doc__) 

    # Authors: Yann N. Dauphin, Vlad Niculae, Gabriel Synnaeve 
    # License: BSD 

    import numpy as np 
    import matplotlib.pyplot as plt 

    from scipy.ndimage import convolve 

    from keras.models import Sequential 
    from keras.layers.core import Dense,Activation 
    from keras.wrappers.scikit_learn import KerasClassifier 
    from keras.utils import np_utils 

    from sklearn import datasets, metrics 
    from sklearn.model_selection import train_test_split 
    from sklearn.neural_network import BernoulliRBM 
    from sklearn.pipeline import Pipeline 


    #%% 
    # Setting up 

    def nudge_dataset(X, Y): 
     """ 
     This produces a dataset 5 times bigger than the original one, 
     by moving the 8x8 images in X around by 1px to left, right, down, up 
     """ 
     direction_vectors = [ 
      [[0, 1, 0], 
      [0, 0, 0], 
      [0, 0, 0]], 

      [[0, 0, 0], 
      [1, 0, 0], 
      [0, 0, 0]], 

      [[0, 0, 0], 
      [0, 0, 1], 
      [0, 0, 0]], 

      [[0, 0, 0], 
      [0, 0, 0], 
      [0, 1, 0]]] 

     shift = lambda x, w: convolve(x.reshape((8, 8)), mode='constant', 
             weights=w).ravel() 
     X = np.concatenate([X] + 
          [np.apply_along_axis(shift, 1, X, vector) 
          for vector in direction_vectors]) 
     Y = np.concatenate([Y for _ in range(5)], axis=0) 
     return X, Y 

    # Load Data 
    digits = datasets.load_digits() 
    X = np.asarray(digits.data, 'float32') 
    X, Y = nudge_dataset(X, digits.target) 
    X = (X - np.min(X, 0))/(np.max(X, 0) + 0.0001) # 0-1 scaling 

    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, 
                 test_size=0.2, 
                 random_state=0) 

    #%% 
    def create_model(): 

     model = Sequential() 
     model.add(Dense(100, input_dim=64)) 
     model.add(Activation('tanh')) 

     """ 
     #other layer 
     model.add(Dense(500)) 
     model.add(Activation('tanh')) 
     """ 

     model.add(Dense(10)) 
     model.add(Activation('softmax')) 
     # Compile model 
     model.compile(loss = 'binary_crossentropy', optimizer = 'adadelta', metrics=['accuracy']) 
     return model 

    rbm = BernoulliRBM(random_state=0, verbose=True) 

    #This is the model you want. it is in sklearn format 
    clf = KerasClassifier(build_fn=create_model, verbose=0) 

    classifier = Pipeline(steps=[('rbm', rbm), ('VNN', clf)]) 

    #%% 
    # Training 

    # Hyper-parameters. These were set by cross-validation, 
    # using a GridSearchCV. Here we are not performing cross-validation to 
    # save time. 
    rbm.learning_rate = 0.06 
    rbm.n_iter = 20 
    # More components tend to give better prediction performance, but larger 
    # fitting time 
    rbm.n_components = 64 

    #adapt targets to hot matrix 
    yTrain = np_utils.to_categorical(Y_train, 10) 
    # Training RBM-Logistic Pipeline 
    classifier.fit(X_train, yTrain) 

    #%% 
    # Evaluation 

    print() 
    print("NN using RBM features:\n%s\n" % (
     metrics.classification_report(
      Y_test, 
      classifier.predict(X_test)))) 

    #%% 
    # Plotting 

    plt.figure(figsize=(4.2, 4)) 
    for i, comp in enumerate(rbm.components_): 
     plt.subplot(10, 10, i + 1) 
     plt.imshow(comp.reshape((8, 8)), cmap=plt.cm.gray_r, 
        interpolation='nearest') 
     plt.xticks(()) 
     plt.yticks(()) 
    plt.suptitle('64 components extracted by RBM', fontsize=16) 
    plt.subplots_adjust(0.08, 0.02, 0.92, 0.85, 0.08, 0.23) 

    plt.show() 

Answer 2

您需要包裝你Keras模型作爲Scikit首先學習模型，然後照常進行。

下面是一個簡單的例子（我略去了進口）

這是一個完整的博客文章有許多其他的例子：Scikit-learn Pipeline Examples

# create a function that returns a model, taking as parameters things you 
# want to verify using cross-valdiation and model selection 
def create_model(optimizer='adagrad', 
       kernel_initializer='glorot_uniform', 
       dropout=0.2): 
    model = Sequential() 
    model.add(Dense(64,activation='relu',kernel_initializer=kernel_initializer)) 
    model.add(Dropout(dropout)) 
    model.add(Dense(1,activation='sigmoid',kernel_initializer=kernel_initializer)) 

    model.compile(loss='binary_crossentropy',optimizer=optimizer, metrics=['accuracy']) 

    return model 

# wrap the model using the function you created 
clf = KerasRegressor(build_fn=create_model,verbose=0) 

# create parameter grid, as usual, but note that you can 
# vary other model parameters such as 'epochs' (and others 
# such as 'batch_size' too) 
param_grid = { 
    'clf__optimizer':['rmsprop','adam','adagrad'], 
    'clf__epochs':[4,8], 
    'clf__dropout':[0.1,0.2], 
    'clf__kernel_initializer':['glorot_uniform','normal','uniform'] 
} 

# just create the pipeline 
pipeline = Pipeline([ 
    ('clf',clf) 
]) 

pipeline.set_params(param_grid) 

pipeline.fit(X_train, y_train)