flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: MultipleLinearRegression - Strange results
Date Thu, 04 Jun 2015 14:51:33 GMT
I agree that given a small data set it's probably better to solve the
linear regression problem directly. However, I'm not so sure how well this
performs if the data gets really big (more in terms of number of data
points). But maybe we can find something like a sweet spot when to switch
between both methods. And maybe a distributed conjugate gradient methods
can also beat SGD if the data is too large to be computed on a single
machine.

Until we have adagrad or another more robust learning rate strategy, we
could also deactivate the default value for simple SGD. This makes users
aware that they have to tweak this parameter.

On Thu, Jun 4, 2015 at 2:54 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> On Thu, Jun 4, 2015 at 1:26 PM, Till Rohrmann <trohrmann@apache.org>
> wrote:
>
> > Maybe also the default learning rate of 0.1 is set too high.
> >
>
> Could be.
>
> But grid search on learning rate is pretty standard practice. Running
> multiple learning engines at the same time with different learning rates is
> pretty plausible.
>
> Also, using something like adagrad will knock down high learning rates very
> quickly if you get a nearly divergent step. This can make initially high
> learning rates quite plausible.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message