readdir/telldir/seekdir problem (i think)

From: Julian Elischer <julian_at_freebsd.org>
Date: Thu, 23 Apr 2015 11:20:37 +0800
I'm debugging a problem being seen with samba 3.6.

basically  telldir/seekdir/readdir don't seem to work as advertised..

here's a little test program

#include <stdio.h>
#include <sys/types.h>
#include <dirent.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sysexits.h>
#include <err.h>

#define CHUNKSIZE 5
#define TOTALFILES 40

static void
SeekDir(DIR *dirp, long loc)
{
     printf("Seeking back to location %ld\n", loc);
     seekdir(dirp, loc);
}

static long
TellDir(DIR *dirp)
{
     long loc;

     loc = telldir(dirp);
     printf("telldir assigned location %ld\n", loc);
     return (loc);
}

int
main(int argc, char *argv[])
{
     DIR            *dirp;
     int        i;
     int        j;
     long        offset = 0, prev_offset = 0;
     char           *files[100];
     char        filename[100];
     int        fd;
     struct dirent  *dp = NULL;

     if (chdir("./test2") != 0) {
         err(EX_OSERR, "chdir");
     }

     /*****************************************************/
     /* Put a set of sample files in the target directory */
     /*****************************************************/

     for (i=1; i < TOTALFILES ; i++)
     {
         sprintf(filename, "file-%d", i);
         fd = open(filename, O_CREAT, 0666);
         if (fd == -1) {
             err(EX_OSERR, "open");
         }
         close(fd);
     }
     dirp = opendir(".");
     offset = TellDir(dirp);
     for (i = 0; i < 20; i++)
         files[i] = malloc(20);

     /*******************************************************/
     /* enumerate and delete small sets of files, one group */
     /* at a time.                                          */
     /*******************************************************/
     do {

         /*****************************************/
         /* Read in up to CHUNKSIZE file names    */
         /* i will be the number of files we hold */
         /*****************************************/
         for (i = 0; i < CHUNKSIZE; i++) {
             if ((dp = readdir(dirp)) != NULL) {
                 strcpy(files[i], dp->d_name);

                 printf("readdir (%ld) returned file %s\n",
                     offset, files[i]);

                 prev_offset = offset;
                 offset = TellDir(dirp);

             } else {
                 printf("readdir returned null\n");
                 break;
             }
         }


/****************************************************************/
         /* Simuate the last entry not fitting into our (samba's) 
buffer */
         /* If we read someting in on the last slot, push it 
back        */
         /* Pretend it didn't fit. This is approximately what SAMBA 
does.*/
/****************************************************************/
         if (dp != NULL) {
             /* Step back */
             SeekDir(dirp, prev_offset);
             offset = TellDir(dirp);
             i--;
             printf("file %s returned\n", files[i]);
         }

         /*****************************************/
         /* i is the number of names we have left.*/
         /*  Delete them.                         */
         /*****************************************/
         for (j = 0; j < i; j++) {
             if (*files[j] == '.') {
                 printf ("skipping %s\n", files[j]);
             } else {
                 printf("Unlinking file %s\n", files[j]);
                 if (unlink(files[j]) != 0) {
                     err(EX_OSERR, "unlink");
                 }
             }
         }
     } while (dp != NULL);

     closedir(dirp);
     //chdir("..");

}

The program is simulating what Samba does when fails. (doing a 
recursive delete of a directory)
What it does is reads a chunk of names using readdir() until it's 
(small) buffer s full,
then it uses seekdir() to seek back before the last entry it read, 
(which fails to fit),
  theortically leaving it for the next read.
It then deletes the entries it found and repeats the cycle.

Eventually it should have found all the files in the directory and 
deleted them.
Except that it doesn't.

What actually happens is that some files are not enumerated, even though
the seekdir() should have made the readdir() find them.
for added fun. the FIRST seekdir appears to work. but all subsequent 
ones don't.

It behaves this way in -current , all the way back to at least 8.0.

if there's a bug in my program please let me know, but samba has the 
same problem.. e.g. on freeNAS.

to use the program make a directory called "./test2" and then run it 
in the current directory..
It fills it with files and then tried to  (fails) delete them in small 
batches.

here's some (annotated) output:

./testit
telldir assigned location 0
readdir (0) returned file .
telldir assigned location 1
readdir (1) returned file ..
telldir assigned location 2
readdir (2) returned file file-1
telldir assigned location 3
readdir (3) returned file file-2
telldir assigned location 4
readdir (4) returned file file-3
telldir assigned location 5
    >>>>> here we pretend the buffer was full and put the file
    >>>>> marker back so that it will get read next time
Seeking back to location 4
telldir assigned location 4
file file-3 returned
skipping .
skipping ..
Unlinking file file-1
Unlinking file file-2
readdir (4) returned file file-3
    >>>>> hey it worked (this time)
telldir assigned location 5
readdir (5) returned file file-4
telldir assigned location 6
readdir (6) returned file file-5
telldir assigned location 7
readdir (7) returned file file-6
telldir assigned location 8
readdir (8) returned file file-7
telldir assigned location 9
    >>>>> OK do it again.. pretend file-7 didn't fit..
    >>>>> set the pointer back so we re-read it next time.

Seeking back to location 8
telldir assigned location 8
file file-7 returned
Unlinking file file-3
Unlinking file file-4
Unlinking file file-5
Unlinking file file-6
    >>>>> OK lets go get file-7 again

readdir (8) returned file file-9
    >>>>>  WTF?  what happened to file-7 ?

telldir assigned location 9
readdir (9) returned file file-10
telldir assigned location 10
Received on Thu Apr 23 2015 - 01:20:48 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:57 UTC