Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

[Pádraig Brady]

On 26/02/10 22:55, Joey Degges wrote:
> Hello,
>
> I added calls to the new function xfadvise wherever a file is opened for
> reading. From what I could tell files are only opened for reading from
> within stream_open and open_temp so that's where I added xfadvise calls.
>
> xfadvise calls posix_fadvise with the flags NOREUSE, SEQUENTIAL, and
> WILLNEED.
>
> I tested this patch with ~500M non-cached files from both flash and disk
> drives. To ensure the files were not cached I remounted the device before
> each run. I found that the variance in the disk drive measurements was too
> high to draw any conclusions (+/- 7seconds). On the flash drives the
> variance was much lower and I saw a consistent speed up of 1 second.
>
> Can you offer any suggestions/comments?

Thanks for the patch.
The speed up of 1s was what % as a matter of interest?
I presume the drive was the bottleneck and the CPU was not at 100%?

> +/* Announce the access patterns NOREUSE, SEQUENTIAL, and WILLNEED on the
> +   descriptor FD. Ignore any errors -- this is only advisory.  */
> +
> +static void
> +xfadvise (int fd)
> +{
> +#if HAVE_POSIX_FADVISE
> +  posix_fadvise (fd, 0, 0, POSIX_FADV_NOREUSE);
> +  posix_fadvise (fd, 0, 0, POSIX_FADV_SEQUENTIAL);
> +  posix_fadvise (fd, 0, 0, POSIX_FADV_WILLNEED);
> +#endif
> +}
> +

http://lxr.linux.no/#linux+v2.6.33/mm/fadvise.c#L27
On the latest linux kernel one can see from the above that...

SEQUENTIAL doubles the read ahead size which seems useful because
by definition sort needs to read all of its input before outputting anything.

NOREUSE used to do the same as WILLNEED, but currently does nothing.
I guess it could bump down the cache priority of the specified range.

I'm not sure WILLNEED has any effect when passed a 0 len?
It's mainly useful I'd say for apps specifying sections
to read ahead which could be a large gain for mechanical
disks and a subsequent random access pattern.

It's interesting that the increased read ahead didn't
improve anything for your mechanical disk. I suppose the
default read ahead values are tuned well there already.
Also it seems like your SSD benefits from the larger read ahead.
This is echoed in this recent post:
http://article.gmane.org/gmane.linux.kernel.mm/43753
which suggests that we soon might get the speedup on SSDs
without apps needing to specify SEQUENTIAL?
See also: http://lkml.org/lkml/2010/2/3/27

So in summary, sequential access is handled quite well
automatically by linux at least, but this might help.
Drop the other 2 settings for now I think.

Codewise you might want probably want to do:
ignore_value (posix_fadvise (fd, 0, 0, POSIX_FADV_SEQUENTIAL));

cheers,
Pádraig.