Re: (problem found) grantpt broken on -CURRENT

On Tue, 2005-07-05 at 00:57 -0400, Joe Marcus Clarke wrote:
> I recently discovered the gnome-vfs sftp method which uses
> posix_openpt/ptsname/grantpt to set up a pty for communicating with the
> sftp subprocess fails to work.  I found that grantpt() fails due to
> EACCES.  The underlying reason looks to be that the pty name returned by
> ptsname() is not unhidden in devfs.  Basically, the following code will
> fail:
> 
> #include <stdlib.h>
> #include <fcntl.h>
> #include <stdio.h>
> #include <errno.h>
> 
> main(void) {
>         int fd;
>         char *p;
> 
>         fd = posix_openpt (O_RDWR | O_NOCTTY);
> 
>         if (fd < 0) {
>                 printf("Failed to open PTY: %s\n", strerror(errno));
>                 return -1;
>         }
>         else {
>                 if ((p = ptsname(fd)) != NULL) {
>                         printf("ptsname = %s\n", p);
>                 }
>                 else {
>                         printf("Failed to get ptsname: %s\n", strerror(errno));
>                         close (fd);
>         if (grantpt(fd) < 0) {
>                 printf("Failed to run grantpt: %s\n", strerror(errno));
>                 close (fd);
>                 return -1;
>         }
> 
>         close (fd);
> 
>         return 0;
> }
> 
> Basically, everything works until the grantpt() call which returns
> EACCES.  The same code works just fine under 5-STABLE.  I even tried
> manually applying devfs rules to unhide the missing tty device
> (/dev/ttys5 in my tests), but that didn't work.  The program also fails
> in the same way when run as root.
> 
> This has been failing since I upgraded from 5-STABLE to -CURRENT about
> three weeks ago.  I'm now running -CURRENT from yesterday.

I did some more research, and found that the problem was not in devfs
hiding the devices, but rather in the way ptsname() determined the tty
name for the open file descriptor.  The kernel gets the minor number for
a tty from the si_drv0 member of the device cdev structure.  However,
ptsname() relies on the minor number of the st_rdev member of the stat
structure to get the tty name.  Devfs sets the st_rdev value to the
device's inode xor a random number.  Therefore, ptsname() returns
garbage, and grantpt() subsequently fails.

I created this patch that corrects the problem, but I'm not sure if it's
the right way to go.  Poul-Henning (or anyone) could you have a look and
let me know if this is okay?  I'd really like to see this working again
in -CURRENT so 6.0 GNOME users will be able to use their sftp shares.
Thanks.

Oh, and it might be a good idea to make sys/types.h aware of MAXMINOR as
it still hardcodes the constant.

Joe

-- 
Joe Marcus Clarke
FreeBSD GNOME Team      ::      gnome_at_FreeBSD.org
FreeNode / #freebsd-gnome
http://www.FreeBSD.org/gnome