Re: cp: behavior regression in 8.23

[Pádraig Brady]

On 30/01/15 11:11, TAMUKI Shoichi wrote:
> Hello,
> 
> I upgraded coreutils from 8.22 to 8.23 and when I copied a directory
> using cp command with -a option, I noticed a behavior regression of
> the copy operation.
> 
> This issue is reproducible on ext[234] w/o dir_index feature, btrfs,
> and xfs filesystems.
> 
> Here is a directory 2014 in ~/work directory:
> 
> tamuki@wombat:~/work$ ls -fliR --fu
> .:
> total 16384
> 1828773 drwxr-xr-x  3 tamuki users 4096 Tue Jan  6 14:10:26 2015 ./
> 1573247 drwx--x--x 65 tamuki users 8192 Tue Jan  6 14:10:09 2015 ../
> 1828789 drwxr-xr-x  2 tamuki users 4096 Tue Jan  6 13:42:44 2015 2014/
> 
> ./2014:
> total 57344
> 1828789 drwxr-xr-x 2 tamuki users 4096 Tue Jan  6 13:42:44 2015 ./
> 1828773 drwxr-xr-x 3 tamuki users 4096 Tue Jan  6 14:10:26 2015 ../
> 1828792 -rw-r--r-- 1 tamuki users    8 Fri Jan 24 14:37:45 2014 January
> 1828810 -rw-r--r-- 1 tamuki users    9 Sat Feb 15 18:05:27 2014 February
> 1828822 -rw-r--r-- 1 tamuki users    6 Sun Mar 23 11:35:16 2014 March
> 1828779 -rw-r--r-- 1 tamuki users    6 Fri Apr 18 12:38:28 2014 April
> 1828797 -rw-r--r-- 1 tamuki users    4 Wed May 14 09:53:07 2014 May
> 1828782 -rw-r--r-- 1 tamuki users    5 Fri Jun 20 11:24:12 2014 June
> 1828824 -rw-r--r-- 1 tamuki users    5 Sat Jul 19 08:42:28 2014 July
> 1828821 -rw-r--r-- 1 tamuki users    7 Tue Aug 12 15:07:39 2014 August
> 1828781 -rw-r--r-- 1 tamuki users   10 Tue Sep  2 22:40:33 2014 September
> 1828793 -rw-r--r-- 1 tamuki users    8 Tue Oct 14 13:52:18 2014 October
> 1828820 -rw-r--r-- 1 tamuki users    9 Fri Nov 28 10:52:36 2014 November
> 1828823 -rw-r--r-- 1 tamuki users    9 Tue Dec 16 13:24:33 2014 December
> 
> Now, copy the directory using cp command with -a option.  The former
> is the cp from 8.22, and the latter is the cp from 8.23.
> 
> tamuki@wombat:~/work$ ~/coreutils-8.22/work/bin/cp -a 2014 coreutils-8.22
> tamuki@wombat:~/work$ ~/coreutils-8.23/work/bin/cp -a 2014 coreutils-8.23
> 
> Here is the result:
> 
> tamuki@wombat:~/work$ ls -fliR --fu
> .:
> total 24576
> 1828773 drwxr-xr-x  5 tamuki users 4096 Tue Jan  6 14:11:39 2015 ./
> 1573247 drwx--x--x 65 tamuki users 8192 Tue Jan  6 14:10:09 2015 ../
> 1828789 drwxr-xr-x  2 tamuki users 4096 Tue Jan  6 13:42:44 2015 2014/
> 2048303 drwxr-xr-x  2 tamuki users 4096 Tue Jan  6 13:42:44 2015 
> coreutils-8.22/
> 2048304 drwxr-xr-x  2 tamuki users 4096 Tue Jan  6 13:42:44 2015 
> coreutils-8.23/
> 
> ./2014:
> total 57344
> 1828789 drwxr-xr-x 2 tamuki users 4096 Tue Jan  6 13:42:44 2015 ./
> 1828773 drwxr-xr-x 5 tamuki users 4096 Tue Jan  6 14:11:39 2015 ../
> 1828792 -rw-r--r-- 1 tamuki users    8 Fri Jan 24 14:37:45 2014 January
> 1828810 -rw-r--r-- 1 tamuki users    9 Sat Feb 15 18:05:27 2014 February
> 1828822 -rw-r--r-- 1 tamuki users    6 Sun Mar 23 11:35:16 2014 March
> 1828779 -rw-r--r-- 1 tamuki users    6 Fri Apr 18 12:38:28 2014 April
> 1828797 -rw-r--r-- 1 tamuki users    4 Wed May 14 09:53:07 2014 May
> 1828782 -rw-r--r-- 1 tamuki users    5 Fri Jun 20 11:24:12 2014 June
> 1828824 -rw-r--r-- 1 tamuki users    5 Sat Jul 19 08:42:28 2014 July
> 1828821 -rw-r--r-- 1 tamuki users    7 Tue Aug 12 15:07:39 2014 August
> 1828781 -rw-r--r-- 1 tamuki users   10 Tue Sep  2 22:40:33 2014 September
> 1828793 -rw-r--r-- 1 tamuki users    8 Tue Oct 14 13:52:18 2014 October
> 1828820 -rw-r--r-- 1 tamuki users    9 Fri Nov 28 10:52:36 2014 November
> 1828823 -rw-r--r-- 1 tamuki users    9 Tue Dec 16 13:24:33 2014 December
> 
> ./coreutils-8.22:
> total 57344
> 2048303 drwxr-xr-x 2 tamuki users 4096 Tue Jan  6 13:42:44 2015 ./
> 1828773 drwxr-xr-x 5 tamuki users 4096 Tue Jan  6 14:11:39 2015 ../
> 2020355 -rw-r--r-- 1 tamuki users    8 Fri Jan 24 14:37:45 2014 January
> 2020357 -rw-r--r-- 1 tamuki users    9 Sat Feb 15 18:05:27 2014 February
> 2020358 -rw-r--r-- 1 tamuki users    6 Sun Mar 23 11:35:16 2014 March
> 2020359 -rw-r--r-- 1 tamuki users    6 Fri Apr 18 12:38:28 2014 April
> 2020360 -rw-r--r-- 1 tamuki users    4 Wed May 14 09:53:07 2014 May
> 2020361 -rw-r--r-- 1 tamuki users    5 Fri Jun 20 11:24:12 2014 June
> 2020362 -rw-r--r-- 1 tamuki users    5 Sat Jul 19 08:42:28 2014 July
> 2020363 -rw-r--r-- 1 tamuki users    7 Tue Aug 12 15:07:39 2014 August
> 2020364 -rw-r--r-- 1 tamuki users   10 Tue Sep  2 22:40:33 2014 September
> 2020365 -rw-r--r-- 1 tamuki users    8 Tue Oct 14 13:52:18 2014 October
> 2020366 -rw-r--r-- 1 tamuki users    9 Fri Nov 28 10:52:36 2014 November
> 2020367 -rw-r--r-- 1 tamuki users    9 Tue Dec 16 13:24:33 2014 December
> 
> ./coreutils-8.23:
> total 57344
> 2048304 drwxr-xr-x 2 tamuki users 4096 Tue Jan  6 13:42:44 2015 ./
> 1828773 drwxr-xr-x 5 tamuki users 4096 Tue Jan  6 14:11:39 2015 ../
> 2020368 -rw-r--r-- 1 tamuki users    6 Fri Apr 18 12:38:28 2014 April
> 2020369 -rw-r--r-- 1 tamuki users   10 Tue Sep  2 22:40:33 2014 September
> 2020370 -rw-r--r-- 1 tamuki users    5 Fri Jun 20 11:24:12 2014 June
> 2020371 -rw-r--r-- 1 tamuki users    8 Fri Jan 24 14:37:45 2014 January
> 2020372 -rw-r--r-- 1 tamuki users    8 Tue Oct 14 13:52:18 2014 October
> 2020373 -rw-r--r-- 1 tamuki users    4 Wed May 14 09:53:07 2014 May
> 2020374 -rw-r--r-- 1 tamuki users    9 Sat Feb 15 18:05:27 2014 February
> 2020375 -rw-r--r-- 1 tamuki users    9 Fri Nov 28 10:52:36 2014 November
> 2020376 -rw-r--r-- 1 tamuki users    7 Tue Aug 12 15:07:39 2014 August
> 2020377 -rw-r--r-- 1 tamuki users    6 Sun Mar 23 11:35:16 2014 March
> 2020378 -rw-r--r-- 1 tamuki users    9 Tue Dec 16 13:24:33 2014 December
> 2020379 -rw-r--r-- 1 tamuki users    5 Sat Jul 19 08:42:28 2014 July
> 
> This affects the result of a creation of tar archive, for example:
> 
> tamuki@wombat:~/work$ tar cpJf monthly.tar.xz coreutils-8.22 coreutils-8.23
> tamuki@wombat:~/work$ tar tvpJf monthly.tar.xz
> drwxr-xr-x tamuki/users      0 2015-01-06 13:42:44 coreutils-8.22/
> -rw-r--r-- tamuki/users      8 2014-01-24 14:37:45 coreutils-8.22/January
> -rw-r--r-- tamuki/users      9 2014-02-15 18:05:27 coreutils-8.22/February
> -rw-r--r-- tamuki/users      6 2014-03-23 11:35:16 coreutils-8.22/March
> -rw-r--r-- tamuki/users      6 2014-04-18 12:38:28 coreutils-8.22/April
> -rw-r--r-- tamuki/users      4 2014-05-14 09:53:07 coreutils-8.22/May
> -rw-r--r-- tamuki/users      5 2014-06-20 11:24:12 coreutils-8.22/June
> -rw-r--r-- tamuki/users      5 2014-07-19 08:42:28 coreutils-8.22/July
> -rw-r--r-- tamuki/users      7 2014-08-12 15:07:39 coreutils-8.22/August
> -rw-r--r-- tamuki/users     10 2014-09-02 22:40:33 coreutils-8.22/September
> -rw-r--r-- tamuki/users      8 2014-10-14 13:52:18 coreutils-8.22/October
> -rw-r--r-- tamuki/users      9 2014-11-28 10:52:36 coreutils-8.22/November
> -rw-r--r-- tamuki/users      9 2014-12-16 13:24:33 coreutils-8.22/December
> drwxr-xr-x tamuki/users      0 2015-01-06 13:42:44 coreutils-8.23/
> -rw-r--r-- tamuki/users      6 2014-04-18 12:38:28 coreutils-8.23/April
> -rw-r--r-- tamuki/users     10 2014-09-02 22:40:33 coreutils-8.23/September
> -rw-r--r-- tamuki/users      5 2014-06-20 11:24:12 coreutils-8.23/June
> -rw-r--r-- tamuki/users      8 2014-01-24 14:37:45 coreutils-8.23/January
> -rw-r--r-- tamuki/users      8 2014-10-14 13:52:18 coreutils-8.23/October
> -rw-r--r-- tamuki/users      4 2014-05-14 09:53:07 coreutils-8.23/May
> -rw-r--r-- tamuki/users      9 2014-02-15 18:05:27 coreutils-8.23/February
> -rw-r--r-- tamuki/users      9 2014-11-28 10:52:36 coreutils-8.23/November
> -rw-r--r-- tamuki/users      7 2014-08-12 15:07:39 coreutils-8.23/August
> -rw-r--r-- tamuki/users      6 2014-03-23 11:35:16 coreutils-8.23/March
> -rw-r--r-- tamuki/users      9 2014-12-16 13:24:33 coreutils-8.23/December
> -rw-r--r-- tamuki/users      5 2014-07-19 08:42:28 coreutils-8.23/July
> 
> So, I would like to fix the problem as following:
> 
> diff --git a/src/copy.c b/src/copy.c
> index 632e119..5179ebb 100644
> --- a/src/copy.c
> +++ b/src/copy.c
> @@ -701,7 +701,7 @@ copy_dir (char const *src_name_in, char const 
> *dst_name_in, bool new_dst,
>    struct cp_options non_command_line_options = *x;
>    bool ok = true;
>  
> -  name_space = savedir (src_name_in, SAVEDIR_SORT_FASTREAD);
> +  name_space = savedir (src_name_in, SAVEDIR_SORT_NONE);
>    if (name_space == NULL)
>      {
>        /* This diagnostic is a bit vague because savedir can fail in

This change was made for performance reasons:
  https://oss.oracle.com/~mason/acp/
  http://home.ifi.uio.no/paalh/publications/files/ipccc09.pdf

What's the particular problem you have with the order
of the files in the tar archive, so I understand your issue completely?

I see that tar 1.28 has the --sort option.
Perhaps if that supported --sort=mtime it would cater for your use case
of reproducible tar archives with a specific order.

thanks,
Pádraig