Re: how to use the ktls

John Baldwin wrote:
>On 1/12/20 8:23 PM, Benjamin Kaduk wrote:
>> On Thu, Jan 09, 2020 at 10:53:38PM +0000, Rick Macklem wrote:
>>> John Baldwin wrote:
>>>> On 1/7/20 3:02 PM, Rick Macklem wrote:
>>>>> Hi,
>>>>>
>>>>> Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS
>>>>> work over TLS.
>>>>> Of course, I know absolutely nothing about TLS, which will make this an interesting
>>>>> exercise for me.
>>>>> I did find simple server code in the OpenSSL doc. which at least gives me a starting
>>>>> point for the initialization stuff.
>>>>> As I understand it, this initialization must be done in userspace?
>>>>>
>>>>> Then somehow, the ktls takes over and does the encryption of the
>>>>> data being sent on the socket via sosend_generic(). Does that sound right?
>>>>>
>>>>> So, how does the kernel know the stuff that the initialization phase (handshake)
>>>>> figures out, or is it magic I don't have to worry about?
>>>>>
>>>>> Don't waste much time replying to this. A few quick hints will keep me going for
>>>>> now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought Kerberos
>>>>> was a pain.;-)
>>>>>
>>>>> Thanks in advance for any hints, rick
>>>>
>>>> Hmmm, this might be a fair bit of work indeed.
>>> If it was easy,  it wouldn't be fun;-) FreeBSD13 is a ways off and if it doesn't make that, oh well..
>>>
>>>> Right now KTLS only works for transmit (though I have some WIP for receive).
>>> Hopefully your WIP will make progress someday, or I might be able to work on it.
>>>
>>>> KTLS does assumes that the initial handshake and key negotiation is handled by
>>>> OpenSSL.  OpenSSL uses custom setockopt() calls to tell the kernel which
>>>> session keys to use.
>>> Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a little
>>> more fun, since it handles TCP connections in the kernel.
>>>
>>>> I think what you would want to do is use something like OpenSSL_connect() in
>>>> userspace, and then check to see if KTLS "worked".
>>> Thanks (and for the code below). I found the simple server code in the OpenSSL doc,
>>> but the client code gets a web page and is quite involved.
>>>
>>>> If it did, you can tell
>>>> the kernel it can write to the socket directly, otherwise you will have to
>>>> bounce data back out to userspace to run it through SSL_write() and have
>>>> userspace do SSL_read() and then feed data into the kernel.
>>> I don't think bouncing the data up/down to/from userland would work well.
>>> I'd say "if it can't be done in the kernel, too bad". The above could be used for
>>> a NULL RPC to see it is working, for the client.
>>
>> So you're saying that we'd only support rpc-over-tls as an NFS client and
>> not as a server, at least until the WIP for ktls read appears?
Actually, I'd say that neither NFS client nor server will work over tls until
the receive side works, since NFS RPCs result in bi-directional traffic.

>To be clear, I have KTLS RX working with TOE right now.  I have a design in my
>head for KTLS RX that would use software and co-processor engines via OCF such
>as aesni(4) and ccr(4) that I hope to implement in the next few months, so KTLS
>RX isn't too far off.  OpenSSL already supports KTLS RX on Linux and the FreeBSD
>patches I already have use the same API.  (Each received TLS frame is read via
>recvmsg() with the TLS header fields in a cmsg.)
Sounds good. It will be a while before I get to the stage where I need it.
I'm currently working  on how to give userland access to a socket created in the
kernel, so that a daemon can use it.

Have fun with it, rick

--
John Baldwin