SP AM relies on the standard TB2 network adapter firmware but does not use any IBM software on the Power2 processor. The adapter firmware allows one user process per node to access a set of memory-mapped send and receive FIFOs directly. These FIFOs allow user-level communication layers to access the network directly without any operating system intervention: the adapter monitors the send FIFO and delivers messages into the receive FIFO, all using DMA for the actual data transfer (Figure 1). The send FIFO has 128 entries while the receive FIFO has 64 entries per active processing node (determined at runtime). Each entry consists of 256 bytes and corresponds to a packet. A packet length array is associated with the send FIFO. Its slots correspond to entries in the send FIFO and indicate the number of bytes to be transferred for each packet. The adapter monitors this length array which is located in adapter memory and transmits a packet when the corresponding slot in the packet length array becomes non-zero.
A packet is sent by placing the data into the next entry of the send
FIFO along with a header indicating destination node and route. Since
the RS/6000 memory bus does not support cache coherency the relevant
cache lines must be flushed out to main memory explicitly. Finally,
the transfer size (1 byte) is stored in the packet length array
located in adapter memory on the microchannel bus. In bulk transfers
this store across the I/O bus can be optimized by writing the lengths
of several packets at a time. To receive a packet, the data in the
top entry of the receive FIFO is copied out to the user buffers.
After being flushed out of the data cache in preparation for a FIFO
wrap-around, the entry is popped from the adapter's receive FIFO.
This is done lazily (after some fixed number of messages polled) to
reduce the number of microchannel accesses (each access costs around
1 
s).