EfficientMixer: A Lightweight Vision Architecture via Complementary Design Principles

作者：Weidong Zhang, Baoxin Li, Huan Liu, Yezhou Yang, Ahmet Arda Dalyanci · 发表于：Preprints.org · 年份：2026 · DOI：10.20944/preprints202607.0512.v1 · 研究领域：Advanced Neural Network Applications、3D Shape Modeling and Analysis、Multimodal Machine Learning Applications

Lightweight vision models are often advanced through architecture-specific innovations or scaling strategies, which can make it difficult to distinguish generalizable design principles from specialized design choices. We hypothesize that lightweight architectures can be more effectively improved by systematically composing complementary components that provide distinct representational capabilities, rather than by optimizing individual building blocks in isolation. Motivated by this perspective, we propose EfficientMixer, a lightweight vision architecture that integrates efficient local feature extraction, spatial mixing, and lightweight global feature modeling within a unified component-level framework inspired by EfficientNet, ConvMixer, and MobileViT. This design enables structured interaction among complementary inductive biases under a fixed budget. Under matched budgets, EfficientMixer consistently outperforms strong lightweight baselines across multiple benchmarks while maintaining similar computational cost and demonstrating improved cross-domain generalization. Furthermore, EfficientMixer achieves larger performance gains under Self-Competitive Distillation, suggesting improved compatibility with advanced training strategies. Extensive ablation studies indicate that these improvements are largely associated with synergistic interactions among complementary architectural components rather than increased model parameters or architectural scaling. Overall, these results...