Re: [lmi] An experimental change for withdrawal solves

[Greg Chicares]

On 2009-02-08 01:53Z, Boutin, Wendy wrote:
[...]
>   File | New | Illustration
>   solve for withdrawal
>     from retirement
>     to maturity
[...otherwise, defaults--except for 'idiosyncrasyT', and...]
>     solve for target $0

[...no surprises so far...]

> Now, just change the gender to male and look at what happens. I
                               ^female^
> see these results in the new 'trace.txt':
> 
>   iteration 21 iterand 61301.6 value 0.2200000000136697
>   iteration 22 iterand 61301.61 value -1.530000000147084
> 
> The output from 'ctrl-d' reveals:
> 
>   61301.6 is level in all years except the last one: 56990.64
>   6528.25 is the cash surrender value in the last year; not good

Reproduced.

> Why is there such a big difference in the withdrawal amounts
> between all years and the last year? And the last year's cash
> surrender value isn't even close to zero, which is inconsistent
> with the output above from 'trace.txt', too.

Sometimes the optimal answer won't produce a CSV close to $0.
(See the footnote.) Yet the solution found doesn't seem optimal.

> Can you please help me understand why there's such an
> inconsistency in the values when all I changed was the
> gender?

I suspect the proximate cause is that the female has a lower COI
rate and a higher withdrawal than the male in the last year, and
that those differences conspired with the $20K premium to mask a
defect for the male in the present experimental implementation.

I suspect the ultimate cause is that the objective function we're
using, as modified 20090202T0247Z, has a zero that isn't actually
the answer we seek [0]. Recall that what we did then:

http://cvs.savannah.gnu.org/viewvc/lmi/lmi/ihs_avmly.cpp?r1=1.112&r2=1.113

was simply to disregard the maximum withdrawal during a solve, so
maybe the maximum withdrawal comes back into play when we make a
"separate final pass" as described in footnote four here:

http://lists.nongnu.org/archive/html/lmi/2009-02/msg00000.html

Before we change the code, what answer would we like it to give?

--- x --  ----- f(x) -----
61301.60  is known to be too high, so let's try...
60000.00  164634.629999999
61000.00  49379.9900000001

What would you guess next? Well, moving from 60000 to 61000 wiped
out about two-thirds of the error, so maybe 61500? A more precise
estimate (by the rule of false position) would be:
  (60000 * 49379.99 - 61000 * 164634.63) / (49379.99 - 164634.63)
  = 61428.4425338537
so let's try that:

--- x --  ----- f(x) -----
61428.44  too high: last year's withdrawal limited
61300.00  still too high, but just barely
61250.00  8449.5899999997
61270.00  too high
61260.00  6811.85999999973
61265.00  I'm too high...
61262.50  ...too high...but I ain't touched the sky
61261.25  6606.829999999
61261.83  too high...but I ain't left the ground
61261.54  6558.81999999807
61261.69  6534.31999999873
61261.76  too high...I can't ever touch the sky
61261.73  ...but no dice... (yet final WD is only 0.55 short)
61261.71  6531.21999999923
61261.72  6529.26999999964

So 61261.72 seems to be the best we can do, if the withdrawal
must be level. A penny higher, and the withdrawal is limited,
though the cash surrender value is a dollar lower. It's a crazy
scene: I don't see how we can ever get it really close to $0.

It is extremely instructive to do this by hand and add a column
showing the difference in the withdrawal between the value you
enter and what you get in the last year. Add 'idiosyncrasyZ' to
the "Comments" field to get monthly details of the calculations,
and compare the "Requested wd" and "Max wd" columns in that final
year. What does that tell you? Does it support my hypothesis, or
lead us to a better one? I had guessed that the absolute amount
of the withdrawal might have something to do with it, but now I'd
say we can dispense with that notion.

Let me present a more pointed hypothesis: that the original male
testcase seemed to succeed because its COI charge in the ultimate
year exceeded the premium that was then paid. The most direct way
to test that is to change "Current COI multiplier" to '1,@99; 0',
then adjust the final-year multiplier until you find a breakpoint
where it spoils the solve. I think it'll be close to 0.8, because
with 1.0, the last year's mortality charge is about 25000, and
eighty percent of that would be the 20000 premium. (What we're
doing here is playing with the function to see how we can make
the discontinuity disappear, to make sure we understand it.)

Today I refactored the loan-solve implementation slightly: see
'ChangeLog' for 20090208T0150Z. Perhaps the approach used for
loans could be made to serve for withdrawals, too.

---------

[0] "a zero that isn't actually the answer we seek"

There's a jump discontinuity, hence no actual zero; what we use
is one of the bounding values. That discontinuity's the four-eyed
cartoon monster on our TV screen.