A transformer-based multi-task deep learning model for urban livability evaluation by fusing remote sensing and textual geospatial data

Livable cities enhance urban economic development, improve physical and mental health, foster well-being, and foster urban sustainability. Evaluating urban livability is therefore important for policymakers to develop urban planning and development strategies aimed at improving livability. Mainstream methods of evaluating urban livability assign different weights to diverse indicators extracted from survey data, statistical data, and geospatial data. To relieve such time-consuming and labor-intensive data collection, this study proposes a transformer-based multi-task multimodal regression (TMTMR) model for the simultaneous evaluation of urban livability focusing on five domain-specific scores. Pretrained state-of-the-art computer vision and natural language processing models serve as backbones to extract features from high spatial resolution remote sensing (RS) images, digital surface models (DSM), night light remote sensing (NLRS) images and point of interest (POI) data. An attention mechanism helps the TMTMR model to assign varying significance levels to features from different modalities, thus capturing both intrinsic information and interrelationships among modalities for livability evaluation. Focusing on 13 Dutch areas, our research demonstrates that the TMTMR model efficiently evaluates urban livability with correlation coefficients ranging from 0.605 to 0.779, and root mean square error values between 0.070 and 0.112 in four unseen test areas. Furthermore, we analyze the synergy between different modalities. We found that modalities of urban livability can be effectively evaluated by aligning, in a descending order, contributions from RS images, NLRS images, DSM, and POI data. We demonstrated that the proposed TMTMR model is capable of effectively evaluating urban livability directly from multimodal geospatial data.