Re: how to use the ktls

John Baldwin wrote:
[stuff snipped]
>I don't know yet. :-/  With the TOE-based TLS I had been testing with, this doesn't
>happen because the NIC blocks the data until it gets the key and then it's always
>available via KTLS.  With software-based KTLS for RX (which I'm going to start
>working on soon), this won't be the case and you will potentially have some data
>already ready by OpenSSL that needs to be drained from OpenSSL before you can
>depend on KTLS.  It's probably only the first few messsages, but I will need to figure
>out a way that you can tell how much pending data in userland you need to read via
>SSL_read() and then pass back into the kernel before relying on KTLS (it would just
>be a single chunk of data after SSL_connect you would have to do this for).
I think SSL_read() ends up calling ssl3_read_bytes(..APPLICATION..) and then it throws
away non-application data records. (Not sure, ssl3_read_bytes() gets pretty convoluted at
a glance.;-)

I've found another issue that should keep me amused for a while (this is becoming an
interesting little project;-).
The KERN_TLS needs unmapped pages on the mbuf chain, but that isn't what NFS
generates.
I think I'll have to implement some sort of copy function that creates mbufs with unmapped
pages and then maps them into kernel space for long enough that the data can be copied,
called just before sosend(). Most NFS RPC messages will easily fit in one page.

Someday, the biggies like server read reply may be able to do what sendfile does and
put the read data in unmapped page mbufs, avoiding the long list of mbuf clusters
that VOP_READ() currently copies the data into.
--> But that's longer term than getting this to work.;-)

Thanks for all your help John, rick

> I'm currently testing with a kernel that doesn't have options KERN_TLS and
> (so long as I get rid of the 478 bytes), it then just does unencrypted RPCs.
>
> So, I guess the big question is.... can I get access to your WIP code for KTLS
> receive? (I have no idea if I can make progress on it, but I can't do a lot more
> before I have that.)

The WIP only works right now if you have a Chelsio T6 NIC as it uses the T6's TCP
offload engine to do TLS.  If you don't have that gear, ping me off-list.  It
would also let you not worry about the SSL_read case for now for initial testing.

--
John Baldwin