--- Begin Message ---
|
Subject: |
[PATCH] cp,mv,install: Disable sparse copy on macOS |
|
Date: |
Thu, 9 Feb 2023 11:20:37 +0200 |
Due to a bug in macOS, sparse copies are corrupted on virtual disks
formatted with APFS. HFS is not affected. Affected are coreutils
install, and gcp when compiled with SEEK_HOLE, as well as macOS Finder.
While reading the entire file returns valid data, scanning for
allocated segments may return holes where valid data is present.
In this case a sparse copy does not contain these segments and returns
zeroes instead. Once the virtual disk is dismounted and then
mounted again, a sparse copy produces correct results.
This breaks OpenWRT build on macOS. Details:
https://github.com/openwrt/openwrt/pull/11960
https://github.com/openwrt/openwrt/pull/11960#issuecomment-1423185579
Signed-off-by: Georgi Valkov <gvalkov@gmail.com>
---
src/copy.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/copy.c b/src/copy.c
index e16fedb28..4f56138a6 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -1065,7 +1065,7 @@ infer_scantype (int fd, struct stat const *sb,
return PLAIN_SCANTYPE;
}
-#ifdef SEEK_HOLE
+#if defined(SEEK_HOLE) && !defined(__APPLE__)
scan_inference->ext_start = lseek (fd, 0, SEEK_DATA);
if (0 <= scan_inference->ext_start || errno == ENXIO)
return LSEEK_SCANTYPE;
--
2.39.1
The issue is most likely within APFS, So we need to bring it to Apple's
attention.
Since it is triggered by building `gcc`, it would be helpful
if they can shine some light how their tools are build
and what trigger this, in order to create PoC code.
**steps to reproduce**
_Create a sparse disk image with APFS_
- open Disk Utility
- press Command-N to create a new virtual disk
- Image Format: sparse disk image
- save as: ~/vhd/test
- Name: test
- Size: 50 GB
- Format: APFS (Case-sensitive)
- Encryption: none
- Partitions: Single partition - GUID Partition Map
- Save
- the image is mounted under `/Volumes/test`
_clone and build coreutils_
``` bash
cd /Volumes/test
# patched with sparse support disabled on macOS
git clone https://github.com/httpstorm/coreutils.git
mv coreutils coreutils-no-sparse
cd coreutils-no-sparse
git checkout macos_disable-sparse-copy
git submodule foreach git pull origin master
./bootstrap
make -j 16
cd ..
# original code
git clone https://github.com/coreutils/coreutils.git
cd coreutils
git submodule foreach git pull origin master
./bootstrap
make -j 16
```
_download OpenWRT and build gcc_
``` bash
cd /Volumes/test
git clone https://github.com/openwrt/openwrt.git
cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a
make defconfig
make menuconfig
# Target System (Marvell EBU Armada)
# Target Profile (Linksys WRT3200ACM)
# Save, Exit
make tools/compile -j 16
make toolchain/gcc/initial/compile -j 16
```
proof of concept
``` bash
cd /Volumes/test/openwrt
cd
build_dir/toolchain-arm_cortex-a9+vfpv3-d16_gcc-12.2.0_musl_eabi/gcc-12.2.0-initial/gcc
/Volumes/test/coreutils/src/cp cc1 cc1-ori
./cc1-ori --help
-bash: ./cc1-ori: cannot execute binary file: Exec format error
/Volumes/test/coreutils-no-sparse/src/cp cc1 cc1-fix
./cc1-fix --help
# works correctly
sha1sum cc1
# fcf5d9458f54d5588efc5a76a89388925ca04eda cc1
sha1sum cc1-fix
# fcf5d9458f54d5588efc5a76a89388925ca04eda cc1-fix
sha1sum cc1-ori
# 7b447132f56f3b549ef62a4780945e82d26f7d2c cc1-ori
```
Kind regards,
Georgi Valkov
httpstorm.com
nano RTOS
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#61386: [PATCH] cp,mv,install: Disable sparse copy on macOS |
|
Date: |
Fri, 24 Feb 2023 22:47:52 +0000 |
|
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0 |
On 24/02/2023 22:06, George Valkov wrote:
If I revert a0803c4bad6f8e11bb05effcc42ef433f4fc3f9b, the requirement to press
enter after PASS: tests/rm/isatty.sh is fixed.
Ah very useful info.
I'll test this on my macOS 13.2 system.
Sometimes it might randomly hang: gdb after
PASS: tests/rm/r-4.sh
^Cmake[1]: *** Deleting file 'tests/rm/r-root.log'
^C^C^C
This is the same gdb issue, which I'll skip similarly
to the tail-2/inotify tests.
Since all issues related to sparse copy on macOS are now addressed,
I'm marking this bug as done.
The above bugs are unrelated, and I'll take them from here.
thanks again for all the testing.
Pádraig
--- End Message ---