MHC: Manifold-Constrained Hyper-Connections for Scalable Residual Models
TL;DR Researchers propose Manifold-Constrained Hyper-Connections (mHC), a framework that constrains hyper-connected residual paths onto a manifold to recover identity-like behavior…