Re: readdir/telldir/seekdir problem (i think)

On Friday, April 24, 2015 06:42:01 PM Julian Elischer wrote:
> On 4/24/15 6:12 AM, Rick Macklem wrote:
> > John Baldwin wrote:
> >> On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote:
> >>> On 4/23/15 11:20 AM, Julian Elischer wrote:
> >>>> I'm debugging a problem being seen with samba 3.6.
> >>>>
> >>>> basically  telldir/seekdir/readdir don't seem to work as
> >>>> advertised..
> >>> ok so it looks like readdir() (and friends) is totally broken in
> >>> the face
> >>> of deletes unless you read the entire directory at once or reset to
> >>> the
> >>> the first file before the deletes, or earlier.
> >> I'm not sure that Samba isn't assuming non-portable behavior.  For
> >> example:
> >>
> >> From
> >> http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html
> >>
> >> If a file is removed from or added to the directory after the most
> >> recent call
> >> to opendir() or rewinddir(), whether a subsequent call to readdir()
> >> returns an
> >> entry for that file is unspecified.
> >>
> >> While this doesn't speak directly to your case, it does note that you
> >> will
> >> get inconsistencies if you scan a directory concurrent with add and
> >> remove.
> >>
> >> UFS might kind of work actually since deletes do not compact the
> >> backing
> >> directory, but I suspect NFS and ZFS would not work.  In addition,
> >> our
> >> current NFS support for seekdir is pretty flaky and can't be fixed
> >> without
> >> changes to return the seek offset for each directory entry (I believe
> >> that
> >> the projects/ino64 patches include this since they are breaking the
> >> ABI of
> >> the relevant structures already).  The ABI breakage makes this a very
> >> non-trivial task.  However, even if you have that per-item cookie, it
> >> is
> >> likely meaningless in the face of filesystems that use any sort of
> >> more
> >> advanced structure than an array (such as trees, etc.) to store
> >> directory
> >> entries.  POSIX specifically mentions this in the rationale for
> >> seekdir:
> >>
> >> http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html
> >>
> >> One of the perceived problems of implementation is that returning to
> >> a given point in a directory is quite difficult to describe
> >> formally, in spite of its intuitive appeal, when systems that use
> >> B-trees, hashing functions, or other similar mechanisms to order
> >> their directories are considered. The definition of seekdir() and
> >> telldir() does not specify whether, when using these interfaces, a
> >> given directory entry will be seen at all, or more than once.
> >>
> >> In fact, given that quote, I would argue that what Samba is doing is
> >> non-portable.  This would seem to indicate that a conforming seekdir
> >> could
> >> just change readdir to immediately return EOF until you call
> >> rewinddir.
> >>
> > Btw, Linux somehow makes readdir()/unlink() work for NFS. I haven't looked,
> > but I strongly suspect that it reads the entire directory upon either opendir()
> > or the first readdir().
> >
> > Oh, and I hate to say it, but I suspect Linux defines the "standard" on
> > this and not POSIX. (In other words, if it works on Linux, it isn't broken;-)
> >
> > rick
> 
> here's an interesting datapoint.  If the test program is run on 
> kFreeBSD using glibc, it runs without flaw.
> 
> OS-X  (bsd derived libc) HFS+ fails
> FreeBSD libc (UFS) fails
> FreeBSD libc (ZFS) fails
> FreeBSD glibc  succceeds
> Centos 6.5 glibc succeeds
> 
> some NFS tests would be nice to do too I guess...
> glibc authors seem to have done something right.. it even copes with 
> FreeBSD kernel..

It's probably just reading the entire directory and caching it until
rewinddir is called.  FreeBSD's libc does this if you have a unionfs
mount.  It would be a trivial change to always do this, is just means
you will never notice any concurrent deletes, adds, or renames until
you call rewinddir again.  At that point you might as well have the
client just do rewinddir each time.  You are just moving the caching that
Samba should be doing to be portable from samba into libc.  I'm not sure
that's really an improvement so much as shuffling deck chairs.

Also, that is going to keep giving you directory entries for the files
you've already removed (unless you patch libc to explicitly hack around
that case by stating each entry and skipping ones that fail with ENOENT
which would add a lot of overhead).

-- 
John Baldwin