混合专家- fminunc优化过早停止

Question

我有一个混合的专家代码，它的工作与少量的参数。其内容如下：

global x_au;
global x_vi;
global x_alpha;
global y;
global parameter;

options = optimoptions(@fminunc,'GradObj', 'on', 'Algorithm','quasi-newton','MaxIter', 10000,'Display','iter-detailed'); % change number of iterations
optTheta=[];

x_au=x_au_train;
x_vi=x_vi_train;
x_alpha=x_alpha_train;
y=y_train;
parameter=zeros(8969,1);

%expectation step
fprintf('opt1 begins');
opt_1;
fprintf('opt1 complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm1=total_error(parameter(1:4483));
k=1;
count = 1;
while(1)
opt_1; 
fprintf('expectation complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm2=total_error(parameter(1:4483));
fprintf('resnorm1-resnorm2 - %f, resnorm2 - %f, k - %f',resnorm1-resnorm2,0.000001*resnorm2,k);
if((resnorm1-resnorm2)< .000001*resnorm2 & k~=1)  %% to decrease training time 
    break;
end

但是现在，当我必须用它来处理大量参数的问题时，我得到了下面的日志。

                                                     First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
                                                   First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 1.000000 
                                                    First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

    resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 2.000000
>> 

然后这个过程就结束了，结果非常糟糕。可以看出，fminunc不能正确地优化。这里有人能帮我吗？

Answer 1

我将参数初始化从零更改为rand，随着规范化的进行，我开始工作。

Answer 2

看起来你需要降低训练速度系数，或者将你的参数向量归一化。