2
我已經編碼了基於這些筆記在Matlab中反向傳播算法：http://dl.dropbox.com/u/7412214/BackPropagation.pdf反向傳播算法（Matlab的）：輸出值被飽和爲1
我的網絡需要長度43的輸入/特徵矢量，已在20個節點隱藏層（我可以改變任意參數選擇），並且具有單個輸出節點。我想訓練我的網絡，使其具有43個特徵，並輸出0到100之間的單個值。將輸入數據標準化爲零均值和單位標準偏差（通過z = x - mean/std），然後附加「1 「輸入向量來表示偏見的術語。我的targetValues只是單個數字0之間和100
這裏是我的代碼的相關部分：
（通過我的慣例，層I（ⅰ）是指輸入層，J（J）是指隱藏層，和K（k）的指的是輸出層，這是在這種情況下，一個單一的節點）
for train=1:numItrs
for iterator=1:numTrainingSets
%%%%%%%% FORWARD PROPAGATION %%%%%%%%
% Grab the inputs, which are rows of the inputFeatures matrix
InputLayer = inputFeatures(iterator, :)'; %don't forget to turn into column
% Calculate the hidden layer outputs:
HiddenLayer = sigmoidVector(WeightMatrixIJ' * InputLayer);
% Now the output layer outputs:
OutputLayer = sigmoidVector(WeightMatrixJK' * HiddenLayer);
%%%%%%% Debug stuff %%%%%%%% (for single valued output)
if (mod(train+iterator, 100) == 0)
str = strcat('Output value: ', num2str(OutputLayer), ' | Test value: ', num2str(targetValues(iterator, :)'));
disp(str);
end
%%%%%%%% BACKWARDS PROPAGATION %%%%%%%%
% Propagate backwards for the hidden-output weights
currentTargets = targetValues(iterator, :)'; %strip off the row, make it a column for easy subtraction
OutputDelta = (OutputLayer - currentTargets) .* OutputLayer .* (1 - OutputLayer);
EnergyWeightDwJK = HiddenLayer * OutputDelta'; %outer product
% Update this layer's weight matrix:
WeightMatrixJK = WeightMatrixJK - epsilon*EnergyWeightDwJK; %does it element by element
% Propagate backwards for the input-hidden weights
HiddenDelta = HiddenLayer .* (1 - HiddenLayer) .* WeightMatrixJK*OutputDelta;
EnergyWeightDwIJ = InputLayer * HiddenDelta';
WeightMatrixIJ = WeightMatrixIJ - epsilon*EnergyWeightDwIJ;
end
end
而且如下權重矩陣被初始化：
WeightMatrixIJ = rand(numInputNeurons, numHiddenNeurons) - 0.5;
WeightMatrixJK = rand(numHiddenNeurons, numOutputNeurons) - 0.5;
%randoms b/w (-0.5, 0.5)
的「乙狀結腸向量「函數採用向量中的每個元素並應用y = 1/(1 + exp(-x))。
這裏的調試消息是什麼樣子，從代碼的開始：
Output value:0.99939 | Test value:20
Output value:0.99976 | Test value:20
Output value:0.99985 | Test value:20
Output value:0.99989 | Test value:55
Output value:0.99991 | Test value:65
Output value:0.99993 | Test value:62
Output value:0.99994 | Test value:20
Output value:0.99995 | Test value:20
Output value:0.99995 | Test value:20
Output value:0.99996 | Test value:20
Output value:0.99996 | Test value:20
Output value:0.99997 | Test value:92
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99998 | Test value:20
Output value:0.99998 | Test value:20
Output value:0.99999 | Test value:20
Output value:0.99999 | Test value:20
Output value:1 | Test value:20
Output value:1 | Test value:62
Output value:1 | Test value:70
Output value:1 | Test value:77
Output value:1 | Test value:20
** stays saturated at 1 **
很明顯，我想網絡訓練輸出值在0和100之間，試圖匹配目標值！
謝謝你的幫助，如果你需要更多的信息我會盡我所能。