bug#26101: Counterproductive calculation order in date

[Assaf Gordon]

Hello Vincent and all,

On 2018-02-06 05:37 PM, Vincent Lefevre wrote:
> Similarly:
>
> zira% date +%Y-%m-%d -d '2003-02-01 - 1 month'
> 2003-01-01
> zira% date +%Y-%m-%d -d '2003-02-01 - 31 days'
> 2003-01-01
>
> but if I add "+ 1 month", I get different results:
>
> zira% date +%Y-%m-%d -d '2003-02-01 - 31 days + 1 month'
> 2003-01-29
> zira% date +%Y-%m-%d -d '2003-02-01 - 1 month + 1 month'
> 2003-02-01
>
> Unfortunately this behavior, which is due to the fact that operations
> are reordered to take into account years, then months, then days, is
> not documented in the Coreutils manual.

That is not accurate - there is no "operation order"
(except in the sense that hours/minutes/seconds are handled separately
after years/months/days).

What happens technically is that when adding years/months/days
the 'date' program operates directly on the "struct tm" member variables 
[1], and then uses the mktime function [2] to normalize invalid dates.

[1] http://pubs.opengroup.org/onlinepubs/7908799/xsh/time.h.html
[2] http://pubs.opengroup.org/onlinepubs/7908799/xsh/mktime.html

In your example:

The date "2003-02-01" becomes the following values in "struct tm":
  tm.tm_year = 103 (years since 1900)
  tm.tm_mon  = 1   (zero-based, 0=Jan, 1=Feb, 2=Mar)
  tm.tm_mday = 1

The date "2003-02-01 - 31 days + 1 month" becomes the following values:

  tm.tm_year = 103
  tm.tm_mon  = 2    (1=Feb + 1 month)
  tm.tm_mday = -30  (1 - 31)

Then calling mktime(3) normalizes it to "2003-01-29".

Please see the attached C program which reproduces the steps above.
The coreutils/gnulib equivalent arithmetics code is in
'parse-datetime.y' line 2161 [3] (or search for the comment "Add 
relative date.").

  [3] 
https://opengrok.housegordon.com/source/xref/gnulib/lib/parse-datetime.y#2161

----

As for "how it should behave",
there are so many edge-cases with date calculations that no single
implementation would satisfy everyone.

To avoid such issues,
it is recommended in the FAQ (and in the --debug output)
to always use the 15th day of the month when adding months
(and similarly, use 12pm when adding days and hours,
and use the middle of year when adding years).



Few examples:

1.
Mar 27th in EEST time zone has only 23 hours due to DST,
which leads to the following difference:

  $ TZ=Europe/Helsinki date -d '2011-03-28 - 24 hours' +%F
  2011-03-26
  $ TZ=Europe/Helsinki date -d '2011-03-28 - yesterday' +%F
  2011-03-27

If you add "12pm" (the middle of the day), then the calculations
are more intuitive:

  $ TZ=Europe/Helsinki date -d '2011-03-28 12pm - yesterday' +%F
  2011-03-27
  $ TZ=Europe/Helsinki date -d '2011-03-28 12pm - 24 hours' +%F
  2011-03-27


2.

February 2003 had 28 days, leading to the following result:

  $ date +%F -d '2003-03-31 - 31 days'
  2003-02-28
  $ date +%F -d '2003-03-31 - 1 month'
  2003-03-03

It becomes clearer with "--debug":

  $ date +%F --debug -d '2003-03-31 - 1 month'
  date: parsed date part: (Y-M-D) 2003-03-31
  [...]
  date: warning: when adding relative months/years, it is recommended to
                 specify the 15th of the months
  [...]
  date: warning: month/year adjustment resulted in shifted dates:
  date:      adjusted Y M D: 2003 02 31
  date:    normalized Y M D: 2003 03 03

But if you had used the 15th of the month, it would "just work":

  $ date +%F -d '2003-03-15 - 1 month'
  2003-02-15

Similarly:

  $ ./date +%F -d '2003-10-30 - 8 months'
  2003-03-02
  $ ./date +%F -d '2003-10-15 - 8 months'
  2003-02-15


3.
months with 30 vs 31 days cause similar issues:

  $ ./date +%F -d '2003-05-30 - 1 month'
  2003-04-30
  $ ./date +%F -d '2003-05-31 - 1 month'
  2003-05-01

  $ ./date +%F --debug -d '2003-05-31 - 1 month'
  date: parsed date part: (Y-M-D) 2003-05-31
  date: parsed relative part: -1 month(s)
  [...]
  date: warning: when adding relative months/years, it is recommended to
                 specify the 15th of the months
  [...]
  date: warning: month/year adjustment resulted in shifted dates:
  date:      adjusted Y M D: 2003 04 31
  date:    normalized Y M D: 2003 05 01
  [...]



4.
And of course, issues with leap years (2016 was a leap year, 2017 was not):

  $ date +%F -d '2016-01-01 + 1 year'
  2017-01-01
  $ date +%F -d '2016-01-01 + 365 days'
  2016-12-31
  $ date +%F -d '2016-02-29 + 1 year'
  2017-03-01

  $ ./date +%F --debug -d '2016-02-29 + 1 year'
  date: parsed date part: (Y-M-D) 2016-02-29
  date: parsed relative part: +1 year(s)
  [...]
  date: warning: month/year adjustment resulted in shifted dates:
  date:      adjusted Y M D: 2017 02 29
  date:    normalized Y M D: 2017 03 01
  [...]


By using the 15th of the month, adding years becomes more intuitive:

  $ date +%F -d '2016-02-15 + 1 year'
  2017-02-15




regards,
 - assaf

date-arith.c
Description: Text Data