bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[jeff.liu]

Jim Meyering wrote:
> jeff.liu wrote:
>> AFAICS, the tests passed on all filesystems except ext4,
> 
> Really?
> The vast majority of my testing is with ext4 on Fedora 14, and I have seen
> no failure -- otherwise I would have mentioned that as a known problem.

I have mentioned this issue at:
http://osdir.com/ml/bug-coreutils-gnu/2010-09/msg00092.html
"make test against cp/sparse-fiemap failed at the extent compare stage, but the 
file content is
identical to each other by comparing those two files "j1/j2" manually.
Is it make sense if we verify them through diff(1) since the testing file is in 
small size?"

> 
> What type of system/kernel are you using?
2.6.33-RC3 && 2.6.36
> Was your ext4 partition created long ago?  With what options?
fiemap copy works well if run `cp' against physical ext4 partition.
> Did "make check" fail?  If so, please provide details.
Yeah, I will show the detail of 'make check' at below.

btw, I just checked out the new branch and tried to compile it but ran into an 
error:
date.c:30:28: error: parse-datetime.h: No such file or directory
date.c: In function 'batch_convert':
date.c:284: warning: implicit declaration of function 'parse_datetime'

I guess 'parse-datetime.h' is shipped with gnulib? For now, I can not pull the 
latest gnulib code
since the remote host does not response.

For a quick reply, I ran 'make check' against the previous code base before 
your latest commit.

sudo make check TESTS=cp/sparse-fiemap VERBOSE=yes

tests/cp/sparse-fiemap.log:
===========================
....
...
+ mount -oloop blob mnt
+ cd mnt
+ echo test
+ test -s f
+ test 0 = 1
+ dd if=/dev/zero of=sparse bs=1k count=1 seek=1G
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 4.6234e-05 s, 22.1 MB/s
+ timeout 10 cp --sparse=always sparse fiemap
++ stat --printf %s sparse
++ stat --printf %s fiemap
+ test 1099511628800 = 1099511628800
+ perl -e 1
++ seq 1 2 21
+ for i in '$(seq 1 2 21)'
+ for j in 1 2 31 100
+ perl -e 'BEGIN { $n = 1 * 1024; *F = *STDOUT }' -e 'for (1..1) { sysseek (*F, 
$n, 1)' -e '&&
syswrite (*F, chr($_)x$n) or die "$!"}'
+ cp --sparse=always j1 j2
+ cmp j1 j2
+ filefrag -v j1
+ grep extent
j1: 2 extents found
+ filefrag -v j1
+ filefrag -v j2
+ f ff1
+ perl /home/jeff/opensource_dev/fiemap_copy/tests/filefrag-extent-compare
+ awk '/^ *[0-9]/ {printf "%d %d ", $2 ,NF < 5 ? $NF : $5 } END {print ""}'
+ sed 's/ [a-z,][a-z,]*$//' ff1
+ f ff2
+ sed 's/ [a-z,][a-z,]*$//' ff2
+ awk '/^ *[0-9]/ {printf "%d %d ", $2 ,NF < 5 ? $NF : $5 } END {print ""}'
@a and @b have different lengths, even after adjustment
+ fail=1
+ break
+ test 1 = 1
+ break
+ Exit 1
+ set +e
+ exit 1
+ exit 1
+ remove_tmp_
+ __st=1
+ cleanup_
+ cd /
+ umount /home/jeff/opensource_dev/fiemap_copy/tests/gt-sparse-fiemap.cXgC/mnt
+ cd /home/jeff/opensource_dev/fiemap_copy/tests
+ chmod -R u+rwx 
/home/jeff/opensource_dev/fiemap_copy/tests/gt-sparse-fiemap.cXgC
+ rm -rf /home/jeff/opensource_dev/fiemap_copy/tests/gt-sparse-fiemap.cXgC
+ exit 1


Thanks,
-Jeff
> If something else failed, please give me enough information
> to reproduce it.
> 
>> but the result is ok by comparing the file
>> contents, can we take this risk?
> 
>> Is this patchset acceptable to merge into the next official release?
> 
> An ext4 failure sounds ominous.
> 
>> Another thing is to add solaris SEEK_DATA support to extent_scan.c as
>> we discussed before, not sure
>> if anyone working on this now. If not, I will take some time to follow
>> up but have to delay about 2
>> weeks since I will on vacation for the chinese new year start from next week.
>>
>> Btw, do you have plan to post extent_scan module to gnulib upstream?
>> so that other file archive
>> projects(like tar(1)) can benefit from it.
> 
> I do not plan to do that right away.
> 
>> Any thing I can do for this patchset please just let me know. :)
> 
> 
>