Empowering real-time traffic reporting systems with NLP-Processed social media data

Current urbanization trends are leading to heightened demand of smarter technologies to facilitate a variety of applications in intelligent transportation systems. Automated crowdsensing constitutes a strong base for ITS applications by providing novel and rich data streams regarding congestion tracking and real-time navigation. Along with these well-leveraged data streams, drivers and passengers tend to report traffic information to social media platforms. Despite their abundance, the use of social media data in ITS has gained more and more attention as of now. In this article, we develop an automated Natural Language Processing (NLP)-based framework to empower and complement traffic reporting solutions by text mining social media, extracting desired information, and generating alerts and warning for drivers. We employ the fine-tuned Bidirectional Encoder Representations from Transformers classification model to filer and classify data. Then, we apply the Question-Answering model to extract necessary information characterizing the reported incident such as its location, occurrence time, and nature of the incidents. Afterwards, we convert the collected information into alerts to be integrated into personal navigation assistants. Finally, we compare the recently posted incident reports from both official authorities and social media in order to provide more complete incident pictures and suggest some open research directions.