A CC Architecture for Host Congestion

hostCC is a congestion control architecture that efficiently handles host congestion, along with network fabric congestion. hostCC provides high network utilization and low latencies in the presence of bottlenecks at the host and/or the network fabric. The key idea in hostCC is using a host-local congestion response operating at a sub-RTT granularity, coupled with network congestion response at RTT granularity.

What is Host Congestion?

  • Host congestion problem. Contention of resources within the host interconnect (like memory bandwidth), leading to PCIe underutilization, NIC queue build-up (and possible packet drops) and network underutilization
  • Origin and trends. Driven by recent adoption of multi-gigabit access link bandwidths, and relatively stagnant technology trends for host resources. Current trends suggest problem is going to get worse with time.
  • Impact. Our HotNets'22 paper characterizes the problem in Google's large-scale production clusters. Our SIGCOMM'23 paper reproduces host congestion phenomenon using open-sourced stacks and CC protocols. We observed >50% throughput degradation, orders of magnitude of latency inflation and >1% packet drop rates at the host.
Host Congestion GIF
Root cause of host interconnect congestion.

hostCC: Key Design Ideas

HostCC Architecture
High-level overview of hostCC architecture.

hostCC embodies three key ideas:
  1. Collecting host congestion signals: hostCC collects congestion signals at the host-interconnect at sub-us granularity. This allows hostCC to capture the precise time, location, and reason for host congestion.
  2. Sub-RTT level host-local congestion response hostCC controls the resources allocated to intra-host traffic (originating from CPUs and other peripheral devices) competing with the network traffic, at a sub-RTT granularity.
  3. Network congestion response using both host and network congestion signals: hostCC performs network resource allocation based on end-to-end bottleneck capacity, also including host-interconnect in addition to each network hop.

hostCC Benefits

Left Image

No HW/SW modifications

Left Image

Improved utilization & latency

Left Image

Interpolates well with network CC


hostCC code and instructions to repoduce its results available at hostCC github repo.

Contribute to hostCC. hostCC architecture does not dictate specific implementations for each of its key design decisions. Feel free to extend hostCC to incorporate (1) additional host congestion signals, (2) additional host resource allocation policies and (3) trying hostCC over additional network CC protocols.


Team Members


Feel free to reach out via email for any questions.