Re: making GNU ls -i (--inode) work around the linux readdir bug

[Jim Meyering]

Ian Jackson <address@hidden> wrote:
> Jim Meyering writes ("Re: making GNU ls -i (--inode) work around the linux 
> readdir bug"):
>> Ian Jackson <address@hidden> wrote:
>> > That is to say you are proposing to fix my complaint by entrenching
>> > the thing I was complaining about.
>>
>> Yes, but only on a system where readdir- and lstat-reported inode
>> numbers differ.
>
> That is all systems.  All UN*X systems since the dawn of time have
> behaved this way.

Just because everyone does it doesn't make it right.
Besides, according to this,
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=369822#60
at least Cygwin 1.5.20 provides a readdir function that works
the way I expect.

>> I think correctness is important enough to sacrifice
>> the optimization in this unusual corner-case usage of ls.
>
> This is the whole _point_ of  ls -i  !

Being fast and inaccurate was never the point of ls -i.
Besides, ls had the "-i" option long before d_ino was invented.

>> (how often do you use ls -i?  of those times, how often
>> are there enough implicitly-listed files that you would
>> notice a longer run time?)
>
> Nearly all of the invocations of  ls -i  on my systems are by
> automated processes which do not list all of the arguments and which
> depend on the optimisation.

If there are tools for which the optimization is important enough,
and ls gets a new option, then people will eventually update them
to use the new option to enable the fast-and-loose
behavior that is currently the default.

Or better still, maybe someone will fix Linux's getdents (the syscall
behind readdir) to do the right thing even in the presence
of mount points.

>> I consider the stat-reported inode numbers to be authoritative.
>> That ls -i prints other numbers _as a result of an optimization_
>> feels disconcertingly like a bug.
>
> I think you should get over this feeling.

Why?

> It's not a bug.

I insist: it is a bug.
If I weren't convinced I wouldn't be spending time on it, now.

> It's the permitted by the specs

The old POSIX spec permitted anything.
The soon-to-be-current version of POSIX has new wording:

    The value of the structure's d_ino member shall be set to the file
    serial number of the file named by the d_name member.

Since "file serial number" is defined, and later associated with
stat.st_ino, that might be enough to cause trouble.  However, I doubt
the austin group realized the implications when they wrote that.

> and it is the way  ls -i  has always behaved in
> every system not using the (broken) coreutils behaviour.

I don't want this behavior to continue being the default, so I plan to
change it, possibly adding an option to enable the stat-free optimization.
But adding new options to ls is a big deal, requiring more justification
than I've seen so far.  If you provide some actual details, like names
of applications, along with performance comparisons, that may be enough.

>> > Note that since ls -i does not print device numbers, the output is not
>> > really meaningful near mountpoints, since inode numbers are only
>> > unique within a device.
>>
>> Perhaps no program relies on 'ls -i'-reported inode numbers
>> (for implicitly-listed files) matching those reported by stat.
>> Not unlikely.  But this is a subtle enough issue that I can
>> imagine it causing trouble some day.
>
> Those programs are already broken on every system which doesn't use
> GNU ls.
>
> The kind of programs which uses  ls -i  is the kind which wants to
> efficiently determine which files are the same as which other files.
> Such programs depend on the knowledge that they aren't going to be
> interfered with by mountpoints.  They also depend on the optimisation
> for acceptable performance.

I don't presume to know all usage scenarios, so want
the default behavior to favor correctness.

>> > All systems have traditionally behaved the way I want: that is, to
>> > return the inode number of the underlying masked mountpoint
>> > directory.
>>
>> I've run experiments on Solaris 10 and FreeBSD 6, and see that they
>> exhibit the same undesirable behavior, so this is not Linux-specific.
>
> So behaviour you consider `undesirable' is in fact the standard.

Ha!  No.  It just means they're all wrong.
Even if POSIX is adjusted or interpreted to allow their legacy
misbehavior, I prefer to make coreutils work around such vagaries.

>> > Really, I don't care what number is returned and neither should anyone
>> > else.  Are there _any_ even arguably correct programs which depend on
>> > the inode number there being `right' ?  What I care about is that
>> > ls -i  should be as fast as readdir.
>>
>> Why?  and more importantly,
>> Why should performance trump correctness?
>
> It's only incorrect in situations where using the inode number is
> incorrect anyway.  You've failed to respond to my comments about the
> lack of the device number.

Sure, but how can ls (in general, and efficiently) know whether
the device number is relevant?
If ls has somehow already done the work to assure that a directory
it is listing contains no mount points, then there's no point
in calling readdir again via ls -i.  If an application "knows"
that the optimization is safe, then it can use the new option.