关键词: 大语言模型, 预测模型, 自注意力机制, 通行费

Abstract:

The prediction of highway tolls is affected by complex factors such as holidays and unexpected events. Traditional prediction methods often fail to fully account for intricate interactions between these multiple factors, resulting in less-than-ideal prediction accuracy. By leveraging the self-attention mechanism, large language models can better fit complex spatiotemporal data and have enhanced feature learning capabilities, making them highly effective for precise highway toll prediction. Therefore, this study proposes a highway toll prediction model based on iTransformer. This model embeds temporal information as an independent dimension into the input sequence and reverses the roles of the self-attention mechanism and feed-forward network, thereby allowing the model to more accurately capture the dynamic features of time series and correlations between multiple variables. Case studies show that the proposed model improves the average prediction accuracy by 23.47% and 17.84% compared with the SARIMA and LSTM models, respectively, in regular scenarios. In irregular scenarios, the model demonstrates even better predictive performance, improving the accuracy by 70.92% and 45.64%, respectively. A sensitivity analysis of the proposed model indicates that it is highly sensitive to the number of feed-forward network layers and stacked encoder layers but is less sensitive to changes in the number of attention heads. Thus, this study provides a new methodological approach for addressing the challenges associated with toll prediction in complex traffic environments and has significant implications in terms of improving the accuracy of highway toll predictions.

Key words: large language model, prediction model, self-attention, toll fee