Improving Parameter Utilization by Sharing Neural Experts Across Layers in Transformers.

arXiv:2609.22199v1 Announce Type: cross Abstract: Transformer-based large language models often suffer from inter-layer parameter redundancy, where functional transformations are redundantly learned across network depths. We propose CS-MoE, a novel Transformer