Single block encoder-decoder transformer model for multi-step traffic flow forecasting

Maslan, Mohd Nazmin and Omar, Masitah Seikh and Yakub, Fitri and Sinha, Vijay Kumar and Abu Talip, Mohamad Sofian and Muljono, Muljono (2024) Single block encoder-decoder transformer model for multi-step traffic flow forecasting. In: 2024 IEEE 6th Symposium on Computers & Informatics (ISCI), 10 August 2024, Kuala Lumpur, Malaysia.

Text Single Block Encoder-Decoder Transformer Model for Multi-Step Traffic Flow Forecasting.pdf Restricted to Registered users only Download (2MB)

Abstract

Accurate traffic flow forecasting is crucial for managing and planning urban transportation systems. Despite the widespread use of sequence modelling models like Long Short-Term Memory (LSTM) for this purpose, the potential of Transformer models remains underexplored. This is particularly true for the simplest form of a single block encoder-decoder Transformer model, which can be finely tuned through optimised hyperparameters. This paper examines the performance of a singular horizon-step forecasting method for multi-step traffic flow forecasting using a proposed Single Block Encoder-Decoder Transformer model optimised with a Grid Search algorithm. Results demonstrate that this model can enhance forecasting accuracy compared to the state-of-the-art LSTM model typically used for multi-step forecasting. The model effectively captures long-range temporal dependencies within a single road traffic flow dataset. It was tested on hourly traffic flow data to forecast the next 24 hours for the I5-North freeway in California, sourced from the Caltrans Performance Measurement System. The optimal configuration included an embedding dimension of 32, a feed-forward dimension of 128, and 8 attention heads. Results show a significant improvement, with a 4.7% reduction in Root Mean Squared Error compared to an LSTM model with two hidden layers of 100 neurons each, showcasing the potential of Single Block Encoder-Decoder Transformer models for real-world traffic prediction applications

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Traffic flow forecasting, Multi-step prediction, Transformer model, Single block encoder-decoder, Long short-term memory
Divisions:	Faculty Of Industrial And Manufacturing Technology And Engineering
Depositing User:	NUR FARISAH JAFRIN
Date Deposited:	23 Jul 2026 00:23
Last Modified:	23 Jul 2026 00:23
URI:	http://eprints.utem.edu.my/id/eprint/29871
Statistic Details:	View Download Statistic

Actions (login required)

View Item