Implement remote socket using RDMA capability for moving files from FTAs to file servers

[brettkettering]

Implement the remote socket using RDMA capability for moving data to/from the component file system's server interacting directly with the ZFS file system and taking NFS out of the loop.

Basic test working. Needs to be integrated to MarFS. Until we have Seagate drivers that allow for writing directly to SMR drives, we need to develop a scheme to maximize writing to the PMR cache and having that transferred to the SMR drives without seeing the big dip in performance we see now if we continue writing full blast to the Zpools. Or, we let the drives themselves gate the data transfer.

Server is now threaded and can handle many clients. Some benchmarking done: 4 threads saturate the IB link at 5.6 GB/s with 1MB payloads. 8 threads don’t get worse.

Getting 200+MB/s on our small zpools. Bandwidth test is taking 10%-20% CPU.

Working on integrating with libne.

Need to add authentication.

Jeff is working on establishing the performance to a capability unit to get an idea of the performance per unit. This will give us some idea about the aggregate performance we should expect from a pod using the remote socket with RDMA mechanism.

Completed the protocol implementation such that the server-side buffer is not overrun while writing and the client-side buffer is not overrun while reading.