Proposal-based visual tracking using spatial cascaded transformed region proposal network

Ximing Zhang, Shujuan Luo, Xuewu Fan

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Region proposal network (RPN) based trackers employ the classification and regression block to generate the proposals, the proposal that contains the highest similarity score is formulated to be the groundtruth candidate of next frame. However, region proposal network based trackers cannot make the best of the features from different convolutional layers, and the original loss function cannot alleviate the data imbalance issue of the training procedure. We propose the Spatial Cascaded Transformed RPN to combine the RPN and STN (spatial transformer network) together, in order to successfully obtain the proposals of high quality, which can simultaneously improves the robustness. The STN can transfer the spatial transformed features though different stages, which extends the spatial representation capability of such networks handling complex scenarios such as scale variation and affine transformation. We break the restriction though an easy samples penalization loss (shrinkage loss) instead of smooth L1 function. Moreover, we perform the multi-cue proposals re-ranking to guarantee the accuracy of the proposed tracker. We extensively prove the effectiveness of our proposed method on the ablation studies of the tracking datasets, which include OTB-2015 (Object Tracking Benchmark 2015), VOT-2018 (Visual Object Tracking 2018), LaSOT (Large Scale Single Object Tracking), TrackingNet (A Large-Scale Dataset and Benchmark for Object Tracking in the Wild) and UAV123 (UAV Tracking Dataset).
Original languageEnglish (US)
Pages (from-to)1-20
Number of pages20
JournalSensors (Switzerland)
Issue number17
StatePublished - Aug 26 2020
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2022-06-14
Acknowledgements: This research was funded by and Ministry of National Defense of China, grant number GFZX04014307 respectively. Thanks to the experimental data provided by University of Ljubljana, SICK, Hiar, King Abdullah University of Science and Technology. Thanks to the experimental facilities provided by Xi’an Institute of Optics and Precision Mechanics of CAS.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

ASJC Scopus subject areas

  • Biochemistry
  • Atomic and Molecular Physics, and Optics
  • Analytical Chemistry
  • Electrical and Electronic Engineering


Dive into the research topics of 'Proposal-based visual tracking using spatial cascaded transformed region proposal network'. Together they form a unique fingerprint.

Cite this