The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

作者：Ma, Shuming, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei · 发表于：arXiv (Cornell University) · 年份：2024 · DOI：10.48550/arxiv.2402.17764 · 被引用次数：37 · 研究领域：Natural Language Processing Techniques、Topic Modeling

End-to-end pipeline for training ternary-weight ({-1, 0, +1}) convolutional neural networks in PyTorch and deploying them as multiply-free inference engines on ESP32-S3 microcontrollers. Validated on MNIST (98.79% accuracy, -0.23 pp drop from FP32) and CIFAR-10 (69.02% accuracy, -6.50 pp drop) with 93.75% weight memory compression and zero multiplications in conv/linear layers. The inference engine is fully dynamic — it reads model dimensions from the binary format at runtime, supporting any input shape without recompilation. Includes PyTorch training scripts, custom .vbn binary export format, zero-dependency C++ desktop engine, and Arduino sketch for ESP32-S3.