Re: Possible bug in NFSv4 with krb5p security?

Thanks for the information. I was looking a bit further into the tcpdump
log, and this is what happens:

Here are some relevant packets:

==== 115 ====
NULL call establishing a mutual context(?)
  GSS-API:
    Kerberos AP-REQ:
      Ticket: Server Name (Principal): nfs/domainname
==== 117 ====
NULL reply to packet 115
==== 169 ====
NFSV4 Call. OPEN DH:.../foo.txt
  Credentials:
    Procedure: RPCSEC_GSS_DATA
    GSS Context: (reference to the context created in the request from
frame 115)
  Verifier:
    GSS Token: (some long sequence of binary data)
    GSS-API: krb5_blob (more binary data)
==== 170 ====
NFSV4 Reply
  OPEN Status: NFS4ERR_ACCESS
=========

The weird thing here is that the OPEN request in packet 169 refers to a
context that was created for the principal name "nfs/domainname" (i.e. the
service principal). Am I correct in my understanding that this context is
different from the user principal that is accessing the data? And if so,
where is the details regarding that principal? Is it in the "verifier" part
in packet 169?

Secondly, what if the issue is gssd not correctly mapping the principals to
Unix usernames? How can I determine if this is the case. There seems to be
no logging options for gssd (-d does absolutely nothing other than prevent
the process from detaching. It still doesn't log anything).

Regards,
Elias

On 13 February 2013 06:47, Rick Macklem <rmacklem_at_uoguelph.ca> wrote:

> Elias Martenson wrote:
> > On 12 February 2013 23:20, Rick Macklem < rmacklem_at_uoguelph.ca >
> > wrote:
> >
> >
> >
> >
> >
> >
> > There is (in case you missed it on google):
> > http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup
> > (Nothing much has changed since FreeBSD8, except the name of the
> > client
> > side patch for host based initiator credentials in the keytab file.)
> > I was hoping others would add/update the wiki and it would eventually
> > become FreeBSD doc, but that hasn't happened.
> >
> >
> >
> >
> > Thank you for the link. I have indeed found that, and I have followed
> > it to the letter.
> >
> >
> > I have up the exact same thing from Ubuntu machines as well as from
> > Solaris, and I do have a fairly good understanding of Kerberos.
> > FreeBSD however, is pretty new to me.
> >
> >
> > Other than that, the various discussions in the archive on this list
> > may help. Unfortunately, I don't know of an easy way to figure out
> > what is busted. I always suggest looking at the packets in wireshark,
> > but for some reason, I get the impression that folk don't like
> > doing this? It is what I do first when I run into NFS issues.
> >
> >
> >
> > I've been looking at the dumps using Wireshark. Well, I had to drop
> > down the security since everything is encrypted when using keb5p. I do
> > get the same errors using sec=krb5.
> >
> >
> > When looking at this trace, I see a normal OPEN request followed by a
> > NFS4ERR_ACCESS as a reply. The Kerberos credentials are of course
> > encrypted, so I can't really say anything about that part.
> >
> Well, it sounds like you are doing all the right stuff, so I don't know
> why it is returning EACCES?
>
> I'm not a ZFS person, so I never test that. If you have a UFS file system
> you could export for testing, that might be worth a try. ZFS likes to do
> things differently;-)
>
> You can look at the authentication stuff in the RPC header:
> Actually, the credentials in the RPC header aren't encrypted, although
> they are binary data. It's been a while since I looked at the RFC, but
> the authenticator is basically:
> - an RPCSEC_GSS version (must be 1 for FreeBSD to be able to use it)
> - a type that will be DATA in this case
> - a credential handle (just a blob of bits the server uses as a shorthand
>     for the principal)
> - a sequence# used to subvert replay attempts
>
> Then the authentication verifier is an encrypted checksum of the above,
> that the server uses to verify it.
>
> All the Kerberos stuff happens via NFS Null RPCs, where the GSSAPI tokens
> are passed as data (a Null RPC has no arguments) and the credential handle
> and a session key get established. (The Kerberos ticket is inside the
> GSSAPI
> token for the first of these Null RPCs.)
>
> >
> > Note that NFS4 with Kerberos security never uses the user ID numbers.
> > They purely use the Kerberos principals for authorisation. This is
> > different from the default "sys" security model that blindly uses user
> > ID's.
> >
> Yep, of course. The Kerberos user principal "name_at_REALM" is translated
> to a uid + gid list by the gssd via a lookup of "name" as the username.
> The uid + gid list is then associated with that credential handle I
> mentioned
> above.
>
> >
> >
> >
> > > nfscbd_enable="YES"
> > Needed for client side only, and only if delegations are
> > enabled at the server and you want the client to acquire
> > delegations. (Delegations are not enabled by default on the
> > FreeBSD NFSv4 server.)
> >
> >
> >
> > Noted. Thank you. I will change this.
> >
> >
> >
> > > rpc_lockd_enable="YES"
> > > rpc_statd_enable="YES"
> > You shouldn't need rpc_lockd or rpc_statd for NFSv4,
> > since they are only used for NFSv3.
> >
> >
> >
> > Good point. I'll disable those too.
> >
> >
> >
> > I don't know why a Linux client would mix NFSv3 RPCs with NFSv4 ones,
> >
> >
> > I was suggested to set vfs.nfsd.server_min_nfsvers to 4 in order to
> > completely disable NFS version below 4. I did this and got rid of the
> > stray NFS3 requests. It didn't solve the original problem though.
> >
> >
> >
> > > If anyone is able to confirm whether or not this actually has been
> > > tested
> > > in 9.1-CURRENT, I'd appreciate it. Also, if not, then I'd love to
> > > know
> > > where I should start looking for a solution. I'm experienced in
> > > system
> > > level programming (having worked on Solaris at Sun in a previous
> > > life), but
> > > a pointer where to start would be helpful.
> > >
> > Usually, when everything is being done by "nobody", it indicates that
> > the mapping between <uid, gids> <-> name_at_domain isn't working
> > correctly.
> > (Looking at the packets in wireshark, you need to look at the
> > attributes
> > called Owner and Owner_Group to see what they are being set to.)
> >
> >
> >
> > I actually doubt this. First of all, I have the correct idmapd setup
> > working from FreeBSD to Ubuntu (I can see that since I can see the
> > correct user names in "ls" even though the user ID's differ). On OSX I
> > haven't got it to work yet.
> >
> >
> > But, the behaviour is the same on both systems.
> >
> >
> > This is actually expected, as the permission checks are orthogonal to
> > the ID mapping.
> >
> >
> > The most common problem (since you do have nfsuserd running on the
> > server)
> > is for the "domain" spec to be different between client and server.
> > FreeBSD's nfsuserd defaults to the domain part of the machine's
> > hostname.
> > Linux's rpc.idmapd sets the domain from /etc/idmapd.conf (at least I
> > think
> > that's what it is called) and many distros ship with it set to
> > "my.domain"
> > or something like that.
> >
> >
> >
> > Correct. I have set this correctly. I know this, since once I did,
> > "ls" started giving me the correct user names.
> >
> >
> > As such, I'd start by checking the Linux client and seeing what it has
> > for
> > the domain spec. in /etc/idmapd.conf.
> >
> > If you want to override the default for FreeBSD, there is a command
> > line
> > option for nfsuserd to do this. I can't remember what it is, but "man
> > nfsuserd"
> > will give you the answer and it can be set in /etc/rc.conf using
> > nfsuserd_flags.
> >
> >
> >
> > Thank you. I'm definitely willing to double-check this and I am not
> > going to claim to know exactly what's going on in the realms of NFS
> > security. :-)
> >
> >
> > If this is configured correctly, then looking at the packets in
> > wireshark is
> > the best starting point w.r.t. figuring out what is broken. I do
> > limited
> > testing of this and it works for me. I don't know how many others use
> > it,
> > although some definitely have fun getting it working. (Usually it is
> > the
> > Kerberos part on the client side that causes the most grief.)
> >
> >
> >
> > It certainly is fun. But it gets frustrating when one fights a single
> > problem for weeks on end.
> >
> >
> > Far too few shops use Kerberos though.
> >
> Or maybe too many;-) A public key system might not be as secure, but it
> would be far easier to manage imho.
>
> >
> >
> > Regards,
> > Elias
>