Skip to content

Status

Jump to bottom
bgoglin edited this page Oct 1, 2012 · 89 revisions

Welcome to the CCI Status page we will track progress weekly here.

UDP LAN Working capabilities: RMA (both read and write), RNR on the recever side, short messages, connection management.

Working on:

  • RNR: Implementation finished, just need to double check with Scott that everything is fine.

  • mss < SOCK_MIN_MSS When the MSS specified during the connection handshake is inferior to the minimum MSS associated to the socket, reject the socket.

Code checking:

  • clang: last ran on 02/01/2012 (UDP branch)

UDP WAN

GNI

Supports cci_connect/cci_accept and associated events. Supports cci_send and associated events. cci_send performance leaves much to be desired. Not sure where problem is... This week will center on fixing the(se) performance issue(s).

OFA Verbs

Updated 2012/01/14

  • Working:

  • Connect, accept, reject

  • Connect full MSG payload

  • MSGs: RO (RU will be ordered since IB is ordered)

  • RMA: Write and Read with/without completion MSG

  • Run through valgrind - all verbs leaks fixed (still 6KB in the ltdl code)

  • Tests: pingpong and stream

  • Reviewed with clang static analyzer

  • ToDo:

  • MSGs: UU provide immediate completion, handle cleanup after transmission

  • MSGS: UU further investigate using UD rather than RC QP

  • Missing: set_opt, get_opt

  • Replace rdma_getaddrinfo to avoid recent lib dependency (maybe)

  • Test fence

  • Many-to-one tests/overrun issues?

  • Handle spurious disconnects, etc.

Ethernet

See the HowTo if you want to use it.

Update 2012/07/29

  • disconnect() support
  • Userspace bookeeping of connections/endpoints
  • poll on OS handle
  • RMA register/deregister
  • Need to tune reliability. Resend comes late, should make it earlier without killing the bandwidth.
  • Reliability adds about +300ns for RU vs UU.
  • Driver works with kernels >= 2.6.32
  • Basic event-delivery through syscalls (list of events in the kernel)
  • Interfaces can be removed/modified by the kernel without breaking everything if endpoints use them

TODO

  • TODO RMA
  • Nack if rhandle invalid or offset+data_len invalid
  • Don't deliver MSG before RMA remotely completed for real
  • RMA MSG seqnum ordered at RMA submit time, not at completion time
  • FENCE flag
  • RMA takes a ref on handle so that deregister doesn't kill things?
  • or keep a list of current RMA in the region handle?
  • when do we create the target RMAop descriptor? special RMAop-init packet? Or when the first packet arrives?
  • abort pending RMA on disconnect()
  • TODO Optimize reliability
  • Force a delayed ack as soon as we have a misorder
  • Immediate ack if still misordered after many other receives?
  • Don't change timer if already scheduled
  • If not piggyacking the bitmap, add a force_ack_with_bitmap flag
  • TODO More checks in connect messages (and MSG): at least req_seqnum
  • TODO bookkeeping of events
  • free (at endpoint close) those that the application didn't return
  • requires locking?
  • TODO set_opt(rx/tx_bufcnt)
  • TODO keepalive_timeout
  • TODO tx_timeout
  • TODO Proper event-delivery through mmapped shared buffer to avoid syscalls
  • TODO Improve ack status and ccieth_pkt_ack_status_to_errno()
  • TODO Optimize cci_sendv()

Open questions

  • Should piggyack in MSG contain the acked bitmap as well?
  • Would simplify management because explicit acks and piggyacks would be the same
  • Should we immediately resend some packets when a selective ack arrives? Only if not resent recently?
  • Resend is currently very slow (0.5s), we don't want to resend to often, it would waste throughput
  • Can we generate events from user-space without breaking future shared-memory mapped ring?
  • At least send and accept events are good candidates
  • If we want to nack MSG when target endpoint or connection doesn't exist, we need the source ep/conn id in all MSG headers
Clone this wiki locally