CE-Prompt: enhance prompt expression stability by multiple understanding

In this article, we propose CE-Prompt, an enhanced version of Prompt-Tuning designed to address issues such as the instability of random initialization and inefficiencies caused by long text in pre-trained large language models (LLMs). Inspired by the multi-head attention mechanism, CE-Prompt introduces the concept of composite embedding, which utilizes multiple randomly initialized embedding layers to generate more expressive prompt representations. To effectively integrate the information expressed by these composite embeddings, an additive fusion approach is employed, allowing each prompt vector to capture task-specific information more comprehensively, thereby improving the model’s task adaptability and inference efficiency. Experimental results show that CE-Prompt outperforms traditional Prompt-Tuning methods, with average improvements of 0.82% in Bilingual Evaluation Understudy (BLEU)-4 and 0.65% in ROUGE-L. Additionally, time complexity analysis indicates that CE-Prompt significantly reduces computational costs during inference. Compared to other methods, it achieves higher efficiency with the same training parameter budget, providing a more efficient solution for practical deployment.