posix_fadvise noreuse disables file caching

Hi,

I recently noticed that multimedia/vlc generates a lot of disk IO when
playing media files. For instance, when playing a 320kbps mp3 gstat
reports about 1250kBps (=10000kbps). That's quite a lot of overhead.

It turns out that vlc sets POSIX_FADV_NOREUSE on the entire file and
reads in chunks of 1028 bytes. FreeBSD implements NOREUSE as if
O_DIRECT was specified during open(2), i.e. it disables all caching.
That means every 1028 byte read turns into a 32KiB read (new default
block size in 9.0) which explains the above numbers.

I've copied the relevant vlc code below (modules/access/file.c:Open()).
It's interesting to see that on OSX it sets F_NOCACHE which disables
caching too, but combined with F_RDAHEAD there's still read-ahead
caching.

I don't think POSIX intended for NOREUSE to mean O_DIRECT. It should
still cache data (and even do read-ahead if F_RDAHEAD is specified),
and once data is fetched from the cache, it can be marked WONTNEED.

Is it possible to implement it this way, or if not to just ignore
the NOREUSE hint for now?


        /* Demuxers will need the beginning of the file for probing. */
        posix_fadvise (fd, 0, 4096, POSIX_FADV_WILLNEED);
        /* In most cases, we only read the file once. */
        posix_fadvise (fd, 0, 0, POSIX_FADV_NOREUSE);
#if defined(HAVE_FCNTL)
        /* We'd rather use any available memory for reading ahead
         * than for caching what we've already seen/heard */
# if defined(F_RDAHEAD)
        fcntl (fd, F_RDAHEAD, 1);
# endif
# if defined(F_NOCACHE)
        fcntl (fd, F_NOCACHE, 1);
# endif
#endif