我已經調整了以下梯度下降算法,用於迴歸存儲在data [:,1]中的x變量中存儲在數據[:,4]中的y變量。然而,梯度下降似乎正在發散。我希望能夠幫助我確定哪裏出錯。爲什麼我的梯度下降R迴歸失敗?
#define the sum of squared residuals
ssquares <- function(x)
{
t = 0
for(i in 1:200)
{
t <- t + (data[i,4] - x[1] - x[2]*data[i,1])^2
}
t/200
}
# define the derivatives
derivative <- function(x)
{
t1 = 0
for(i in 1:200)
{
t1 <- t1 - 2*(data[i,4] - x[1] - x[2]*data[i,1])
}
t2 = 0
for(i in 1:200)
{
t2 <- t2 - 2*data[i,1]*(data[i,4] - x[1] - x[2]*data[i,1])
}
c(t1/200,t2/200)
}
# definition of the gradient descent method in 2D
gradient_descent <- function(func, derv, start, step=0.05, tol=1e-8) {
pt1 <- start
grdnt <- derv(pt1)
pt2 <- c(pt1[1] - step*grdnt[1], pt1[2] - step*grdnt[2])
while (abs(func(pt1)-func(pt2)) > tol) {
pt1 <- pt2
grdnt <- derv(pt1)
pt2 <- c(pt1[1] - step*grdnt[1], pt1[2] - step*grdnt[2])
print(func(pt2)) # print progress
}
pt2 # return the last point
}
# locate the minimum of the function using the Gradient Descent method
result <- gradient_descent(
ssquares, # the function to optimize
derivative, # the gradient of the function
c(1,1), # start point of theplot_loss(simple_ex) search
0.05, # step size (alpha)
1e-8) # relative tolerance for one step
# display a summary of the results
print(result) # coordinate of fucntion minimum
print(ssquares(result)) # response of function minimum
你能分享你的數據嗎? –
這是我正在使用的數據集http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv data < - read.csv(「Advertising.csv」)[, - 1] – esperanto
梯度下降的分歧通常表明學習速率太高,因此降低學習速率(α)以收斂。 –