5 problems

Logarithmic ansatz conjecture. The optimal value function is conjectured to have the form

Let $\lambda_{\texttt{min}}$ denote the minimum visitation probability in the asynchronous Q-learning setting described above. Visitation-probability dependence conjecture. Some de…

Let VRCQ denote the variance-reduced cascade Q-learning algorithm for estimating the optimal Q-function in a discounted Markov decision process, and let $\mathcal{U}$ be its action…

Constant-stepsize asynchronous Q-learning produces iterates whose distributional convergence, convergence-rate characterization, central limit theorem for averaged iterates, asympt…

Risk-aware Q-learning (RaQL) uses an inner--outer loop structure in which the risk is estimated in an inner procedure and the $Q$-values are updated in an outer procedure. RaQL bia…