The FC layer is currently the only mechanism that provides flow control in Horus. It uses a credit (window) based mechanism. Currently, credit is based on the stability information provided by the STABLE layer. When the number of unstable messages in a group exceeds some maximum, the FC layer starts delaying messages. When the number of unstable messages goes below this threshold, the buffered messages are packed together into a single message subject to some maximum number of bytes and sent in a single send operation. Both the maximum number of unstable messages and the maximum number of bytes are adjustable at run-time. (We envision that in future these values will be determined dynamically based on run-time statistics.) Manual adjustment of these parameters shows that this scheme is quite effective, increasing the performance in several important situations.
Note that the flow control layer does not provide any back pressure
(it does not block senders, but instead buffers the messages
that cannot be sent right away). Instead, back pressure is accomplished by
MUTS, which slows down processes that allocate more
memory than they release (we have started working on a paper that describes
this idea in more detail).