Torus interconnect

Diagram of a 3-dimensional torus interconnect. It is not limited to 8 nodes but can consist of any number of nodes in a similar rectilinear array.

A torus interconnect is a network topology for connecting processing nodes in a parallel computer system. It can be visualized as a mesh interconnect with nodes arranged in a rectilinear array of N = 2, 3, or more dimensions, with processors connected to their nearest neighbors, and corresponding processors on opposite edges of the array connected.^[1] The lattice has the topology of an N dimensional torus and each node has 2N connections.

A number of supercomputers on the TOP500 list use three-dimensional torus networks, e.g. IBM's Blue Gene/L and Blue Gene/P, and the Cray XT3.^[2] IBM's Blue Gene/Q uses a five-dimensional torus network. Fujitsu's K computer and the PRIMEHPC FX10 use a proprietary six-dimensional torus interconnect called Tofu.^[3]

Visualization

• In a two-dimensional torus interconnect, the nodes are imagined laid out in a two-dimensional rectangular lattice of rows and columns, with each node connected to its 4 nearest neighbors, and corresponding nodes on opposite edges connected. The connection of opposite edges can be visualized by rolling the rectangular array into a "tube" to connect two opposite edges and then bending the "tube" into a torus to connect the other two.
• In a three-dimensional torus interconnect the nodes are imagined in a three-dimensional lattice in the shape of a rectangular prism, with each node connected with its 6 neighbors, with corresponding nodes on opposing faces of the array connected.

Higher-dimensional arrays can't be directly visualized, but each higher dimension adds another pair of nearest neighbor connections to each node.

While long wrap-around links may be the easiest way to visualize the connection topology, in practice, restrictions on cable lengths often make long wrap-around links impractical. Instead, directly connected nodes -- including nodes that the above visualization places on opposite edges of a grid, connected by a long wrap-around link -- are physically placed nearly adjacent to each other in a folded torus network.^[4][5][6] Every link in the folded torus network is very short -- almost as short as the nearest-neighbor links in a simple grid interconnect -- and therefore low-latency.^[7]

See also

• Computer cluster
• Heartbeat private network
• Switched fabric

References

1. ↑ Industrial Strength Parallel Computing by Alice E. Koniges 1999 ISBN 1-55860-540-1 page 16
2. ↑ N. R. Agida et al. 2005 Blue Gene/L Torus Interconnection Network, IBM Journal of Research and Development, Vol 45, No 2/3 March–May 2005 page 265
3. ↑ Fujitsu Unveils Post-K Supercomputer HPC Wire Nov 7 2011
4. ↑ Cray Inc. "The Gemini Network". 2010. p. 13.
5. ↑ "Small-World Torus Topology".
6. ↑ Pavel Tvrdik. "Topics in parallel computing: Embeddings and simulations of INs: Optimal embedding of tori into meshes".
7. ↑ "The 3D Torus architecture and the Eurotech approach".