Warner Losh wrote: >On Fri, Feb 26, 2021 at 11:16 AM Rodney W. Grimes < >freebsd-rwg_at_gndrsh.dnsmgr.net> wrote: > >> > On Fri, Feb 26, 2021 at 9:24 AM Rodney W. Grimes < >> > freebsd-rwg_at_gndrsh.dnsmgr.net> wrote: >> > >> > > > My understanding is that KTLS works very well with OpenSSL for >> sending, >> > > but >> > > > not as well for receiving, because there's nothing like a recvfile >> > > > syscall. However, it works great for both send and receive with NFS, >> > > where >> > > > all the data remains in the kernel. What about zfs recv? A very >> common >> > > > pattern is for an application to read from an SSL socket and then >> pipe >> > > the >> > > > data to zfs recv. For example, zrepl does that. Could zfs recv >> instead >> > > > read directly from the KTLS socket, bypassing userspace? That could >> > > > potentially save a _lot_ of cycles for a _lot_ of people. >> > > >> > > I did some patches and a short presentation at BSDCan that basically >> > > shoves the whole zfs send and zfs recv process into the kernel, ie >> > > it opens the sockets up, makes the connections, then the socket >> > > is passed into the kernel(s) and it all runs in kernel mode. >> > > >> > > >> > > >> https://www.bsdcan.org/2018/schedule/attachments/479_BSDCan-2018-zfs-send.pdf >> > > >> > > A few things need fixed like reversing who does the listen for >> > > security reasons, but this feature is probably ready for prime >> > > time. >> > > >> > > > -Alan >> > > >> > > -- >> > > Rod Grimes >> > > rgrimes_at_freebsd.org >> > >> > >> > That looks potentially useful, but it doesn't use encryption. Would it >> > work if the socket had been opened by openssl with ktls? >> >> Yes, it should. Internally the zfs send and recv code just does reads >> and writes to the socket, so what ever you setup for "connected" sockets >> should work. >> > >Yea, KTLS generally wants userland to do the initial negotiation and share >the connection state before doing the bulk encryption in the kernel... Yes. KTLS only handles application data records. The TLS handshake must be done in userspace. Handling of non-application data records received after the initial handshake also needs to be handled in userspace. All the userspace program does is an SSL_read() with a len argument == 0. This will always return an error, but only after it has read the non-application data records off the head of the socket's receive queue. The kernel code needs to tell the userspace program to do this somehow. The nfs-over-tls code does this via an upcall RPC to the userspace daemon (rpc.tlsclntd or rpc.tlsservd). I added MSG_TLSAPPDATA which can be used as an argument to soreceive() to tell it to return an error when there is a non-application data record at the head of the socket's receive queue, to facilitate this. You do normal sosend()/soreceive() on the socket with unencrypted data, although each soreceive() will give you a ctrlmsg mbuf with TLS record info, if you care (when you use MSG_TLSAPPDATA, they're all just application data records). Each soreceive() gives you one application data record, but the code probably won't care. Finally, when done transferring data, userspace probably wants to do SSL_shutdown() to do the peer reset (although if/when you need to do peer resets seems to be a bit sketchy in the TLS game). rpc.tlsclntd.c and rpc.tlscommon.c probably gives you some useful hints. It sounds worthwhile to explore this. Good luck with it, rick Warner _______________________________________________ freebsd-current_at_freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"Received on Fri Feb 26 2021 - 23:11:12 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:27 UTC