Re: bfgsmin and fminunc

[Jaroslav Hajek]

On Mon, May 18, 2009 at 10:29 AM, Michael Creel <address@hidden> wrote:
>
> Hi Jaroslav,
> Thanks for the explanation, and I'm glad to see optimization getting
> some attention. I'm not too well informed, but what I have observed
> suggests that the trust region approach is used more widely in more
> recent work.  I don't know too much about it, though, and I can't
> comment on the relative merits compared to line search. I would say
> that there's no reason not to combine the two if it leads to a
> flexible, robust, and reliable algorithm.  I agree that the optim
> toolbox could use some work. My contributions there are bfgsmin and
> samin. I would like to modify them to have an interface more in line
> with the "industry standard". Right now, these algorithms allow for
> multiple arguments to the function to be minimized, passed as a cell,
> and the one to minimize with respect to can be specified. That works
> well for me, but it is not standard.

Yes, in some cases that approach is convenient, and in the past it was
even necessary to avoid global variables, but I now consider it
superseded by anonymous function handles, which provide a more general
and flexible convenient mechanism to accomplish the same:

fminunc (@(x) my_func(x, y, z), x0,....)

the *big* advantage here is that the argument selection and binding
part is kept out of the function; fminunc just deals with a single
function of a single argument and stays simple. Thus, this mechanism
works exactly the same for all optimization, integration and any other
functions. The disadvantage, at least for now, is somewhat lower
performance, but I have plans for improvements here.


> I need to study the optimset
> mechanism, I still don't know how it works. I will try to spend some
> time this summer on this.

Basically, Octave's optimset/optimget now maintain internally a list
of "registered options" (see __all_opts__).
When you set an option via optimset, it looks up the list and possibly
adds proper casing, otherwise it warns you if an unregistered option
is being set, such as when you type "GradientObj" instead of "GradObj"
or so. Options are registered at Octave's startup using a "default"
query for each registered function. In a package, newly registered
function are added using PKG_ADD. See Octave's fsolve, fminunc, fzero
and lsqnonneg. Other functions (sqp, qp, glpk) either do not yet
conform to this interface, or do not need it (qp).

> If you could have a look at __bfgsmin.cc for
> improvements, I would appreciate it. With an eye to improvements, I
> believe that there is probably some low hanging fruit in that code.
> Cheers, Michael
>

I plan to do some work here, but probably not until Octave 3.2 gets out.

cheers


-- 
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz