Real-Time Tracking in a Pull-Based Status Update System With Unreliable Command Channel

We consider a pull-based status update system consisting of a finite-state Markov source, an energy-harvesting-enabled transceiver, and a sink. Both data and command channels are error-prone. The sink is interested in real-time tracking of the source and sends commands for updates to the transceiver. However, due to noise in the command channel, the transceiver may experiencefalse alarmsormiss-detections. We study the problem of minimizing the long-term time average of a (generic) distortion subject to a constraint on the average number of commands. Due to the pull-based nature of the system, the controller (i.e., sink) has only partial observability of the source state and the available energy at the transceiver. Therefore, we model the problem as a constrained partially observable Markov decision process (C-POMDP), which is then cast as a constrained belief-MDP (C-belief-MDP) problem. The infinite belief-state space makes solving the C-belief-MDP difficult. Thus, we effectively truncate the belief-state space and formulate a finite-state C-belief-MDP problem. We develop a stationary randomized policy (SRP) to solve the problem by employing Lagrangian relaxation, the relative value iteration algorithm (RVIA), and bisection search. Due to the curse of dimensionality in deriving the SRP, we also develop a low-complexity policy applying the drift-plus-penalty method, which transforms the C-belief-MDP problem into a sequence of per-slot optimization problems. Simulation results show the effectiveness of the proposed policies and their superiority compared to baseline policies.