Re: [Octave Forge] Possible addition of gradient-descent framework for optim package

[Olaf Till]

Hi Daniel,

On Fri, Oct 23, 2015 at 07:57:57AM +0200, Daniel Kraft wrote:
> (I sent this already to Olaf some time ago, but I think it makes sense
> to give everyone a possible say in the discussion.)

the following reply was made up (and not sent) one month ago when I
suggested you to involve the list with this.

Though there are also standalone optimizers in 'optim', there are also
frontends, whose interfaces were attempted to be designed as general
as possible. If possible, new algorithms should not be standalone, but
backends to the existing frontends. Your so_... functions would
introduce a different interface, which would be a complication to
'optim' which outweighs, in my opinion, the possible advantages.

As for the algorithms: Gradient descent is not what I'd think
desirable for 'optim'. Line search is e.g. implemented in core Octaves
'sqp' with a few lines of code, also Armijo with backtracking, but
additionally constraints are handled, which would be necessary for a
general algorithm employing line search. I wouldn't think a public
function is needed for line search, since it is typically used as a
component of another algorithm.

You mention the possibility of storing and replaying the course of
optimization. The existing frontends of 'optim' can call user-provided
callbacks, with which it should be possible to implement this. Can't
you interface the existing functions of 'optim' with your package by
providing suitable callbacks, instead of writing separate optimization
code? If some functionality for this should be lacking in 'optim', we
could possibly add it.

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature