Re: Algorithmic Differentiation in Octave

[Brad Bell]

On 01/24/2017 11:03 AM, Olaf Till wrote:
> On Tue, Jan 24, 2017 at 08:35:09AM -0700, Brad Bell wrote:
> > I have created a link from Cppad
> >      https://www.coin-or.org/CppAD/
> > to Octave (and some other languages) through Swig. See
> >      http://www.seanet.com/~bradbell/cppad_swig/cppad_swig.htm
> > Also see the getting started example for Octave
> > http://www.seanet.com/~bradbell/cppad_swig/a_fun_jacobian_xam.m.htm
> >
> > Feel free to contact me if you have any questions about it.
> >
> > Brad.
> I thought of implementing AD myself for Octave some time... The
> natural concept seems to me to have classes whose objects
> (representing vectors) you can give as arguments to a general scripted
> function (restriction: function must only use operations which are
> overloaded for the object) making this function return the
> derivatives.

I think you are referring what is called tapeless forward mode AD; see
    https://goo.gl/rqc2Dl

>   Your concept seems to be different, and more complicated
> for application -- it seems to involve creation of a separate
> representation of the function code ('ay(0) = ax_0 * ax_1 * ax_2; af =
> m_cppad.a_fun(ax, ay);' in your example)...?
Yes, the function is taped (recorded). Once it is taped, it can be 
optimized, sparsity patterns can be computed, and derivatives, or all 
orders, and that take advantage of sparsity, can be calculated. In 
addition, the function can be evaluated (with C++ like speed) at 
different points (provided that the operation sequence does not depend 
on the independent variables):
https://www.coin-or.org/CppAD/Doc/glossary.xml#Operation.Sequence
> For an important application (optimizing dynamic models which are
> solved by providing sensitivity equations) we'd also need second
> derivatives... And for efficiency, it would be good to be able to mark
> some second derivatives which are not needed...
>
> Olaf