bug#16539: More details on df command output for you

[Pádraig Brady]

On 01/27/2014 01:06 AM, Pádraig Brady wrote:
> On 01/26/2014 11:35 PM, Bernhard Voelker wrote:
>> On 01/26/2014 12:28 PM, Pádraig Brady wrote:
>>> On 01/25/2014 11:55 PM, Bernhard Voelker wrote:
>>>> However, I remember some other corner cases with eclipsed file
>>>> systems in the Fedora bug tracker. I think we're quite close
>>>> to solve them all this time (hopefully).
>>>> The idea was to trust the order of mount entries returned by
>>>> the kernel, i.e. in the loop over the mount entries, if the
>>>> mount point is the same one as a previous one, then we should
>>>> process the one mounted later.
>>>>
>>>> E.g. the situation where 2 file systems are mounted on the
>>>> same mount point:
>>>>
>>>>   $ findmnt | grep loop
>>>>   └─/mnt                           /dev/loop0     ext4                
>>>> rw,relatime,data=ordered
>>>>     └─/mnt/dir                     /dev/loop1     ext4                
>>>> rw,relatime,data=ordered
>>>>       └─/mnt/dir                   /dev/loop2     ext4                
>>>> rw,relatime,data=ordered
>>>>
>>>> df - the new one with your patch - still shows the wrong device:
>>>>
>>>>   $ src/df | grep loop
>>>>   /dev/loop0        122835      1551    112110   2% /mnt
>>>>   /dev/loop1        122835      1550    112111   2% /mnt/dir
>>>>
>>>> It should say /dev/loop2 here. BTW the numbers are correct.
>>
>> BTW: the fstype is wrong, too (which can only be seen with -T or --output,
>> and if it differs, of course).
>>
>>> Right, that could be handled easy enough.
>>> loop1 is not accessible above and so should be hidden.
>>> But consider a bind mount resulting in something like:
>>>
>>>>   └─/mnt                           /dev/loop0     ext4                
>>>> rw,relatime,data=ordered
>>>>     └─/mnt/dir                     /dev/loop1     ext4                
>>>> rw,relatime,data=ordered
>>>>       └─/some/place/else           /dev/loop1     ext4                
>>>> rw,relatime,data=ordered
>>>>       └─/mnt/dir                   /dev/loop2     ext4                
>>>> rw,relatime,data=ordered
>>>
>>> If we did a linear scan through that, we'd lose the /some/place/else
>>> due to it being a longer mount dir, and then also the original loop1
>>> as we took /dev/loop2 for /mnt/dir.
>>> Seems like when discarding we would need to see if this was the
>>> last entry for a device and then see if there are any other candidate
>>> mount points for that device?
>>
>> Hi Padraig,
>>
>> thanks.
>> Again, mount_list is a little beast - more below.
>>
>> The following patch (on top of yours) would handle both cases
>> without a problem.  Feel free to squash it in, if you like.
>>
>> diff --git a/src/df.c b/src/df.c
>> index 23b5156..78768cc 100644
>> --- a/src/df.c
>> +++ b/src/df.c
>> @@ -631,9 +631,20 @@ filter_mount_list (void)
>>        else
>>          {
>>            /* If we've already seen this device...  */
>> +          struct devlist *d = NULL;
>>            for (devlist = devlist_head; devlist; devlist = devlist->next)
>>              if (devlist->dev_num == buf.st_dev)
>> -              break;
>> +              {
>> +                d = devlist;
>> +                if (!STREQ (devlist->me->me_devname, me->me_devname))
>> +                  {
>> +                    /* Fix the devname if the mount dir has been
>> +                       mounted over by a different devname.  */
>> +                    free (devlist->me->me_devname);
>> +                    devlist->me->me_devname = xstrdup (me->me_devname);
>> +                  }
>> +              }
>> +          devlist = d;
>>
>>            if (devlist)
>>              {
>>
>> But there is yet another issue with the -a mode for such
>> over-mounted and therefore eclipsed file systems:
>>
>>   # Create 2 file system images: 1 ext4, 1 xfs.
>>   $ dd if=/dev/zero bs=1M status=none count=128 of=img1
>>   $ dd if=/dev/zero bs=1M status=none count=256 of=img2
>>   $ mkfs -t ext4 -F img1 >/dev/null 2>&1
>>   $ mkfs -t xfs  -f img2 >/dev/null 2>&1
>>   $ mkdir /mnt{1,2}
>>
>>   # Mount both on /mnt1.
>>   $ mount -o loop img1 /mnt1
>>   $ mount -o loop img2 /mnt1
>>
>>   # Mount the former (ext4) also on /mnt2 via its loop device.
>>   $ mount /dev/loop0 /mnt2
>>
>>   # Result:
>>   $ findmnt --output=TARGET,SOURCE,FSTYPE | grep loop
>>   ├─/mnt1                          /dev/loop0     ext4
>>   │ └─/mnt1                        /dev/loop1     xfs
>>   └─/mnt2                          /dev/loop0     ext4
>>
>> Everything is fine now with the filtered df run ...
>>
>>   $ src/df --out -h | grep loop
>>   /dev/loop1     xfs        256K     3  256K    1%  252M   13M  239M   6% -  
>>   /mnt1
>>   /dev/loop0     ext4        32K    11   32K    1%  120M  1.6M  110M   2% -  
>>   /mnt2
>>
>> ...but "df -a" prints the wrong statistics for the "over-mounted" /mnt1!
>>
>>   $ src/df --out -h -a | grep loop
>>   /dev/loop0     ext4                  256K     3  256K    1%  252M   13M  
>> 239M   6% -    /mnt1
>>   /dev/loop1     xfs                   256K     3  256K    1%  252M   13M  
>> 239M   6% -    /mnt1
>>   /dev/loop0     ext4                   32K    11   32K    1%  120M  1.6M  
>> 110M   2% -    /mnt2
>>
>> Okay, this is nothing new.
>> BTW: strictly speaking, also the output of today's "df -t rootfs -a"
>> is wrong because the numbers are definitely not that of the early-boot
>> rootfs file system.
>>
>> Now, how should df handle this?
>>
>> a)
>> df silently filters out the mount entries of all eclipsed mount dirs,
>> even with -a.
>> --> Hmm, I think this would probably contradict to POSIX.
>>
>> b)
>> df prints an error diagnostic for each eclipsed mount dir, and exits
>> non-Zero.
>> --> Well, there are probably such mounts on every system, e.g. on my box:
>>
>>   TARGET                       SOURCE         FSTYPE
>>   /proc/sys/fs/binfmt_misc     systemd-1      autofs
>>   /proc/sys/fs/binfmt_misc     binfmt_misc    binfmt_misc
>>
>> Therefore, a "df -a" would always fail. ;-(
>> At least on my system, there are
>>
>> c)
>> df prints a warning diagnostic for each eclipsed mount dir, and exits
>> Zero (unless another error occurs).
>>
>> --> Due to the same reason as in b), these warning might be messy
>> and users will probably be irritated.
>>
>> d)
>> df outputs "-" for all numbers of such eclipsed file systems, e.g.
>>
>>   $ src/df --out -h -a | grep mnt1
>>   /dev/loop0     ext4                     -     -     -     -     -     -    
>>  -    - -    /mnt1
>>   /dev/loop1     xfs                   256K     3  256K    1%  252M   13M  
>> 239M   6% -    /mnt1
>>
>>
>> Maybe d) is the best solution, as it mirrors what df can know:
>> it knows source, target and the file system type, but it doesn't
>> have access to the block and inode numbers.
>>
>> WDYT?
> 
> Thanks for the nice analysis and tests.
> d) seems like the best option here, though we'd have to be careful
> about cases where /proc/mounts was giving a system wide view,
> while df wasn't privy to that due to mount namespaces or
> overmounts etc. I'm not thinking of a specific issue here,
> just the general problem.
> 
> wrt c) and annoying warnings, I also notice `df -a` on a default Fedora 20 
> install here,
> giving multiple duplicate warnings like:
>   df: ‘net:[4026532416]’: No such file or directory
>   df: ‘net:[4026532416]’: No such file or directory
> That's due to:
>   $ grep net: /proc/mounts
>   proc net:[4026532416] proc rw,nosuid,nodev,noexec,relatime 0 0
>   proc net:[4026532416] proc rw,nosuid,nodev,noexec,relatime 0 0
> Which is due to support for namespaces.
> Seems like we should not try to lookup non absolute mount points?

Hi Bernhard,

I've attached 4 patches for df to:

    df: also deduplicate virtual file systems
    df: fix handling of symlinks in mount list
    df: ignore non file system entries in /proc/mounts
    maint: avoid clang -Wtautological-constant-out-of-range-compare warning

Not included is your "overmount change" or the "d)" adjustment above,
as I was unsure how you wanted to handle exactly.

thanks,
Pádraig.

df-8.23-fixes.patch
Description: Text Data