Improving Sequential Recommendations with Token-Level LLM Representation

作者：Dingzhou Wang, Chang Lu, Luyao Men, Jin He, Yinuo Yang, Yefeng Liang · 年份：2025 · DOI：10.36227/techrxiv.176290955.59211749/v1 · 被引用次数：2 · 研究领域：Recommender Systems and Techniques、Explainable Artificial Intelligence (XAI)、Topic Modeling

Sequential recommendation systems often use ID-based embeddings, which are efficient but lack semantic richness and generalization. Large language models (LLMs) can capture contextual information, yet their use in recommendation is limited. We propose a model that initializes recommendation sequences with token-based LLM representations, transferring linguistic knowledge into item representations. Unlike traditional embeddings, our method applies subword and contextual encoding to preserve semantic detail across diverse items. On benchmarks like Amazon-Books and MovieLens-1M, our approach achieves higher accuracy with less memory and training time than SASRec, BERT4Rec, and GPT-based recommenders. Ablation studies further show faster convergence and reduced overfitting. These results demonstrate that LLM token-based initialization is an efficient and cost-effective paradigm for advancing sequential recommendation.