Re: readdir/telldir/seekdir problem (i think)

On 4/23/15 9:54 PM, John Baldwin wrote:
> On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote:
>> On 4/23/15 11:20 AM, Julian Elischer wrote:
>>> I'm debugging a problem being seen with samba 3.6.
>>>
>>> basically  telldir/seekdir/readdir don't seem to work as advertised..
>> ok so it looks like readdir() (and friends) is totally broken in the face
>> of deletes unless you read the entire directory at once or reset to the
>> the first file before the deletes, or earlier.
> I'm not sure that Samba isn't assuming non-portable behavior.  For example:
>
> >From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html
>
> If a file is removed from or added to the directory after the most recent call
> to opendir() or rewinddir(), whether a subsequent call to readdir() returns an
> entry for that file is unspecified.
>
> While this doesn't speak directly to your case, it does note that you will
> get inconsistencies if you scan a directory concurrent with add and remove.
>
> UFS might kind of work actually since deletes do not compact the backing
> directory, but I suspect NFS and ZFS would not work.

looking in ufs_getdirent (or whatever it's called) even UFS is packing
the list of files and removing deleted entries.
it's also what I'm seeing in practice.
if I do a seekdir back to the location of the first deleted file, I 
find a new file in that slot,
that has been shuffled down.

>   In addition, our
> current NFS support for seekdir is pretty flaky and can't be fixed without
> changes to return the seek offset for each directory entry (I believe that
> the projects/ino64 patches include this since they are breaking the ABI of
> the relevant structures already).  The ABI breakage makes this a very
> non-trivial task.  However, even if you have that per-item cookie, it is
> likely meaningless in the face of filesystems that use any sort of more
> advanced structure than an array (such as trees, etc.) to store directory
> entries.  POSIX specifically mentions this in the rationale for seekdir:
>
> http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html
>
> One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once.
>
> In fact, given that quote, I would argue that what Samba is doing is
> non-portable.  This would seem to indicate that a conforming seekdir could
> just change readdir to immediately return EOF until you call rewinddir.
>